Filtering mass data
I do have 50k raw data, and need do filtering data.
We have tried use :
1. Key and keyboards, and mixture with press window. the results didn't accurate.
2. Excel with IF function, it costs us around 2 hours to check the data.
3. Vb script doesn't work at our environment. And we don't want to keep Marco file for running the filtering data.
Is that any possibility or guide by using powershell? Powershell since more workable to our environment.
Use Exec SQL action with SQL Select against your large Excel. It is very fast.
Each Worksheet Tab name is considered to be the From Table.
Only requirement is to put the column / header names in the 1st row of the Excel
You can do sql select, insert, updates.
You need to install the Microsoft ACE Driver for Exec SQL against Excel.
Search the forum for articles downloading and setting up the Driver
P.S. For any future VBScriot need, there is a built-in Run VBScript action that holds the vbscript internally in an action, for execution
On my blog you have an example working with 100.000 lines (in my case, a list from a csv file).
Executing a sum operation on each line takes around 36 seconds. You can see some tools I have used, as Joseph recommends using some text/sql driver you can do your task instantly.
another method is to use some language to process/parse your task, I love to use some basic compiler like powerbasic or libertybasic, both really cheap.
1 person likes this