ES 1.1 Command Line Tool for Everything 1.4.1

[void]

Introducing ES 1.1 the command line interface to Everything 1.4.1

ES-1.1.0.29.x86.zip
ES-1.1.0.29.x64.zip



Usage:

• es.exe [options] search text

Standard ES command line options.

• These options will continue to function with older versions of Everything.

-r <search>, -regex <search>

• Search using regular expressions.

-i, -case

• Match case.

-w, -ww, -whole-word, -whole-words

• Match whole words.

-p, -match-path

• Match full path and file name.

-h, -help

• Display this help.

-n <num>, -max-results <num>

• Limit the number of results shown to <num>.

-s

• sort by full path.

The following ES features require Everything 1.4.1 or later.

Column types

-name
-path
-full-path-and-name
-extension, -ext
-size
-date-created, -dc
-date-modified, -dm
-date-accessed, -da
-attributes, -attribs, -attrib
-file-list-file-name
-run-count
-date-run
-date-recently-changed, -rc

• Show the specified column, if path, name or full path and name is not specified, full path and name is used.
  You can specify which columns you would like ES to display and in which order they are displayed

Sorting

-sort name[-ascending|-descending], -sort-name[-ascending|-descending]
-sort path[-ascending|-descending], -sort-path[-ascending|-descending]
-sort size[-ascending|-descending], -sort-size[-ascending|-descending]
-sort extension[-ascending|-descending], -sort-extension[-ascending|-descending]
-sort date-created[-ascending|-descending], -sort dc[-ascending|-descending], -sort-dc[-ascending|-descending]
-sort date-modified[-ascending|-descending], -sort dm[-ascending|-descending], -sort-dm[-ascending|-descending]
-sort date-accessed[-ascending|-descending], -sort da[-ascending|-descending], -sort-da[-ascending|-descending]
-sort attributes[-ascending|-descending], -sort-attributes[-ascending|-descending]
-sort file-list-filename[-ascending|-descending], -sort-file-list-filename[-ascending|-descending]
-sort run-count[-ascending|-descending], -sort-run-count[-ascending|-descending]
-sort date-recently-changed[-ascending|-descending], -sort rc[-ascending|-descending], -sort-rc[-ascending|-descending]
-sort date-run[-ascending|-descending], -sort-date-run[-ascending|-descending]

• Sort results.
  Note: Works best with fast sort types.

-sort-ascending, -sort-descending

• Set sort order

/on Sort by name ascending.
/o-n Sort by name descending.
/os Sort by size ascending.
/o-s Sort by size descending.
/oe Sort by extension ascending.
/o-e Sort by extension descending.
/od Sort by date modified ascending.
/o-d Sort by date modified descending.

• DIR style sorting

General options

-instance <name>

• Connect to the unique Everything instance name.

-o <offset>, -offset <offset>

• Show results starting from offset.

-export-csv <out.csv>
-export-efu <out.efu>
-export-txt <out.txt>
-export-m3u <out.m3u>
-export-m3u8 <out.m3u8>
-export-tsv <out.tsv>

• Export to a file.
  Exporting as csv will export all specified columns.
  Exporting as efu will export the filename, size, date modified, date created and attributes columns.
  Exporting as txt will export only the filename column.
  Exporting as m3u will export as ANSI play list.
  Exporting as m3u8 will export as UTF-8 play list.
  Exporting as tsv will export all specified columns.

-csv
-efu
-txt
-m3u
-m3u8
-tsv

• Change display format.
  Output can be redirected.

-no-header

• Don't write the CSV or EFU header.

-utf8-bom

• Write a UTF-8 byte order mark at the start of the export file.

-size-format <format>

• 0=auto, 1=Bytes, 2=KB, 3=MB.

-pause, -more

• Pause after each page of output.

-hide-empty-search-results

• Hide results when no search is passed to ES.

-empty-search-help

• Show the help when no search is passed to ES.

-timeout <milliseconds>

• Timeout after the specified number of milliseconds to wait for the Everything database to load before sending a query.

-size-leading-zero
-run-count-leading-zero

• Format the column with leading zeros, use with -no-digit-grouping.

-no-digit-grouping

• Don't group numbers with commas.

Colors:

-highlight

• Highlight results.

-highlight-color <color>

• 
  Highlight color 0x00-0xFF.

-filename-color <color>
-name-color <color>
-path-color <color>
-extension-color <color>, -ext-color <color>
-size-color <color>
-date-created-color <color>, -dc-color <color>
-date-modified-color <color>, -dm-color <color>
-date-accessed-color <color>, -da-color <color>
-attributes-color <color>
-file-list-filename-color <color>
-run-count-color <color>
-date-run-color <color>
-date-recently-changed-color <color>, -rc-color <color>

• 
  Highlight color 0x00-0xFF.

Column widths

• By default ES tries to align each column as best as it can, you can customize the width of each column for larger consoles.

-filename-width <width>
-name-width <width>
-path-width <width>
-extension-width <width>, -ext-width <width>
-size-width <width>
-date-created-width <width>, -dc-width <width>
-date-modified-width <width>, -dm-width <width>
-date-accessed-width <width>, -da-width <width>
-attributes-width <width>
-file-list-filename-width <width>
-run-count-width <width>
-date-run-width <width>
-date-recently-changed-width <width>, -rc-width <width>

• Set the minimum column width 0-200.

Search Filters

-path <path>

• Search for subfolders and files in <path>.

-parent-path <path>

• Search for subfolders and files in the parent of <path>.

-parent <path>

• Search for folders and files only in <path>.

/ad

• Directories only.

/a-d

• Files only.

/a[RHSDAVNTPLCOIE]

• DIR style attribute searching.
  R = Read Only
  H = Hidden
  S = System
  D = Directory
  A = Archive
  V = Device
  N = Normal
  T = Temporary
  P = Sparse File
  L = Reparse Point
  C = Compressed
  O = Offline
  I = Not Content Indexed
  E = Encrypted
  - = Prefix a flag with - to exclude.

Saving and clearing settings

-save-settings

• Save all switches to the registry under HKEY_CURRENT_USER\Software\ES

-clear-settings

• Clear all saved settings in the registry under HKEY_CURRENT_USER\Software\ES

• Settings are loaded automatically each time you run ES.
  No search is performed when using -save-settings or -clear-settings, ES will return immediately.

Run history
-set-run-count <filename> <count>

• Set the run count for the specified file to count.
  The file does not have to exist.

-inc-run-count <filename>

• Increment the run count for the specified file by one.

-get-run-count <filename>

• Get the run count for the specified file and display the result.

• No search is performed when using -set-run-count, -inc-run-count or -get-run-count, ES will return immediately.

Result Count
-get-result-count

• Get and display the result count.

• No result file names are shown when using -get-result-count ES will return immediately with the result count.

Limitations

• ES does not have access to bookmarks or filters.

Examples:

• Export all mp3 files to an Everything file list named mp3.efu
  • es.exe *.mp3 -export-efu mp3.efu
  Show the top 10 largest files:
  • es.exe -sort size -n 10
  Show the last 10 modified files:
  • es.exe -sort dm -n 10
  Highlight the search terms foo bar
  • es.exe foo bar -highlight
  Make ES show the size column, date modified column and set some pretty colors and make them the default settings:
  • es.exe -size -dm -sizecolor 0x0d -dmcolor 0x0b -save-settings

Return codes:

Code: Select all

ES_ERROR_SUCCESS                    0 // no known error, search successful.
ES_ERROR_REGISTER_WINDOW_CLASS      1 // failed to register window class
ES_ERROR_CREATE_WINDOW              2 // failed to create listening window.
ES_ERROR_OUT_OF_MEMORY              3 // out of memory
ES_ERROR_EXPECTED_SWITCH_PARAMETER  4 // expected an additional command line option with the specified switch
ES_ERROR_CREATE_FILE                5 // failed to create export output file
ES_ERROR_UNKNOWN_SWITCH             6 // unknown switch.
ES_ERROR_SEND_MESSAGE               7 // failed to send Everything IPC a query.
ES_ERROR_IPC                        8 // NO Everything IPC window - make sure the Everything search client is running.

Notes:

• -'s can be omitted, eg: instead of typing -no-digit-grouping you can type: -nodigitgrouping
  Use double quotes to escape spaces and switches.
  Use ^ to escape special command line characters, eg: escape | with ^|
  Switches can also start with a / eg: /ad
  Switches can be disabled by prefixing them with no-, eg: -no-size.
  Highlighting is expensive, use as many search terms as possible or use -n to limit the number of results.

• Most es features require Everything 1.4.1.
  
  [TODO] screenshots
  
  Source code is included. ES 1.1 uses the new Everything 1.4 IPC.
  
  Please let me know if you have any suggestions or comments, thanks.

[therube]

At first the board wasn't displaying correctly, so I wasn't able to see the page "clearly".
Then what seemingly should have been easy to accomplish, says its going to need more study.

But on a quick look, I think I'm see oddities?

Code: Select all

es -size -dm -sizecolor 13 -sort -dm -n 10

OK, so if you get the syntax wrong, you get odd results.

Better:

Code: Select all

es -size -dm -sizecolor 13 -sort dm -n 10

[void]

I should make ES print an error message when the sort type is unknown..

[therube]

RFE: Allow -path to use relative paths.

Code: Select all

ES  -path  .   *.db
ES  -path  ..\test  *.lz

I have long had an "SS.BAT" that I would use after dropping to a C: prompt from within my file manager to search for something within a particular tree:

Code: Select all

@dir /a /b /p /o:n /s *%1* %2 %3

If ES allowed relative, I could easily replace DIR with ES.
(Not really a big deal in this case, usually the starting point (.) of the tree isn't that deep [or high?].)


PS:

Code: Select all

ES  -path  .\  *.db

seems to cause a crash.

And after said crash, attempting to run ES returns:

Code: Select all

Everything IPC window not found, IPC unavailable.

So it actually killed Everything.exe (GUI) too.
[I'm on 7.80b, at present.]
(The Everything.exe service does persist.)

[keithg1964]

Very nice additional options.

One item I see missing from my wishlist is the ability to pause after a certain number say 25. If I use the more.com command, it losses the color effects.

Looking forward the the color table, as your color options look strange on my black on white window.

Also, I have had kill everything 1.4.1.773b (x64) a couple of times. Is there any info I can provide, or should I upgrade to an newer beta / alpha?

Keith

update1: Seems to kill the program not the service when I search for an file that does not exist.
Update2: Upgraded to 1.4.1.782b (x64) and the crashing is gone.

[therube]

Suppose you automatically switch to ASCII on a redirect or pipe?
Wonder what would happen if you didn't (necessarily) & piped to Less or similar (instead of More)?

[keithg1964]

Bug / Format change request:

Display date as system format instead of DD/mm/yyyy or provide option to change.

Prefer mm/dd/yyyy myself.

[therube]

DD/mm/yyyy

Ah, right.
(Maybe its that way for sort reasons?)

[void]

ES 1.1.0.2a:
http://www.voidtools.com/es-1.1.0.2a.zip

• added -pause
• added -hide-empty-search-results
• added -empty-search-help
• added -save-settings
• added -clear-settings
• added -path <path>
• added -parent-path <path>
• added -parent <path>
• added no- switch prefix to disable switches.
• added localized dates.

If the sort type is unknown, ES will now display an error message
Saving or clearing settings returns immediately, no search is performed.
There was some extra debugging information in 773b that would crash if there was no ES search results, was fixed in 782b.
You can use relative paths with -path <path>
The old -path command line option has been renamed to -path-column, so has -filename to -filename-column.
-pause will pause the display after each page.
dates are now displayed in the correct localized format.
added /ad /a-d command line options.

[void]

Suppose you automatically switch to ASCII on a redirect or pipe?

ES uses WriteConsole when connected to a console, for pipes ES uses WriteFile after converting to the consoles current code page.
No color information is passed to WriteFile.
Color information is only passed to WriteConsole.


Just for reference: chcp can be used to change the current code page of the command prompt.
To change to UTF-8, which maybe useful for piping data to other applications or to a file, use:
chcp 65001

http://superuser.com/questions/269818/c ... e-to-utf-8

[void]

Using -n 10 should probably be treated as a non-empty search, will experiment.

[keithg1964]

Thanks for the pause and the date code formatting change.

The only Issue I am having is the line-wrap on long paths. I have been thinking about how to solve this issue, and the only format that I think would work if you can do it without too much trouble is. Again, if not doable, then no problem.

On paths that exceed available width, then replace the path name after three characters with "..." until the next "\". Depending on full length may need more than one substitution.

Just my thoughts.


Before
325,369 09/19/2016 11:56 C:\Program Files (x86)\Intuit\QuickBooks 2015\Components\Help\conten
t\hlphtml3.zip
966,024 09/19/2016 11:56 C:\Program Files (x86)\Intuit\QuickBooks 2015\Components\Help\conten
t\hlphtml2.zip

After
325,369 09/19/2016 11:56 C:\Pro...\Int...\Qui...\Components\Help\content\hlphtml3.zip
966,024 09/19/2016 11:56 C:\Pro...\Int...\Qui...\Components\Help\content\hlphtml2.zip

[void]

Thanks for the reply, I understand the issue, I'll experiment with ellipses.
If you didn't know you can use -name to show only the name part, but then of-course you lose the path information.
I'm finding I resize the console window so its slighter wider than 80 columns.

[keithg1964]

Yes I know of the -name option. Be sure to either get max columns or allow one to specify max width. My currently is set to 150.

Anther thought, there are so many options, that it takes some time to figure out the correct format. How about an optional configuration dialog box that allows one to select options. The box would have an option to run the search as selected to show output, and an option to either save in registry or to show the current command line option for current config to save as an alias or in a batch file.



Thanks

[jg00d3]

If everything.exe is running as service, can the es.exe -instance parameter be used to specificy the everything.exe service to run or do I have to launch everything.exe UI? I'm trying to run this as a schedule task that is not visible to the user. If I run everything.exe using the -minimized option, the user sees the flashing everything.exe icon in the tray.

Also, is it possible for es.exe to search for multiple files\wildcards in the same search? would it be possible for es.exe to use a config.ini file for it's parameters like everything.exe does? Possible to list the files\wildcards to search for in that config.ini file?

Thanks!

[void]

Anther thought, there are so many options, that it takes some time to figure out the correct format. How about an optional configuration dialog box that allows one to select options. The box would have an option to run the search as selected to show output, and an option to either save in registry or to show the current command line option for current config to save as an alias or in a batch file.

More screenshots might help, I'll be making a Everything wiki page on ES soon..

can the es.exe -instance parameter be used to specificy the everything.exe service

No, ES communicates with the Everything search client.

If I run everything.exe using the -minimized option, the user sees the flashing everything.exe icon in the tray.

Try running Everything.exe with the -startup command line option:
Everything.exe -startup

Also, is it possible for es.exe to search for multiple files\wildcards in the same search?

Yes:
es.exe *.jpg ^| *.png

would it be possible for es.exe to use a config.ini file for it's parameters like everything.exe does?

Instead of saving to the registry? I would have to put the es.ini in %APPDATA%\ES to avoid issues where ES doesn't have write permissions to the folder it is currently in. I could just make it a switch too: -save-ini, or -save-app-data-ini, es could try loading both on startup.

Possible to list the files\wildcards to search for in that config.ini file?

I wanted to avoid saving filters and searches in the settings as it's hard to figure out what es is searching for because they are not shown.

[jg00d3]

Thanks for the info. I can work w/ this but have one last question and then a possible bug report.

what is the maximum number of filename\wildcards that can be searched for at one time. Ex: *.jpg^|*.xls^|*.xlsx (that would be three, is there a limit to how many can be searched for at one time?)

I run everything -minimized, then when I search es.exe *.txt it shows results, but when I search es.exe *.txxt everything.exe crashes. It seems that if there are no search results everything.exe crashes.

Thanks!

[keithg1964]

I run everything -minimized, then when I search es.exe *.txt it shows results, but when I search es.exe *.txxt everything.exe crashes. It seems that if there are no search results everything.exe crashes.

Thanks!

I had the same issue, upgraded to 1.4.1.782b and it was solved.

[therube]

RFE: If -pause, ESC or Q should exit the search.

(And maybe even some more more-like features, like CR to advance 1 line?)

[void]

what is the maximum number of filename\wildcards that can be searched for at one time. Ex: *.jpg^|*.xls^|*.xlsx (that would be three, is there a limit to how many can be searched for at one time?)

ES has a limit of 65535 search characters.
Other than that, there is no limit on how many search terms you have.

ext:jpg;xls;xlsx performance will be much much much greater than *.jpg^|*.xls^|*.xlsx when using multiple extensions.

I run everything -minimized, then when I search es.exe *.txt it shows results, but when I search es.exe *.txxt everything.exe crashes. It seems that if there are no search results everything.exe crashes.

There was some extra debugging information in 773b that would crash if there was no ES search results, was fixed in 782b.

782b should fix the issue.

RFE: If -pause, ESC or Q should exit the search.

(And maybe even some more more-like features, like CR to advance 1 line?)

I'll add the same functionality as DIR|more...

[therube]

Any reason for (more) exit codes?
I gather presently you can get a 0 or something not-0.

0 is fine (checking %errorlevel%), but maybe some other exit conditions could set other levels, that might be of benefit?
(And not that I'm thinking of any at the moment, as a 0-check so far has been sufficient.
I guess you could have something like:
0 success
1 success, no results returned
2 ESC from Pause
3 syntax error
XXX program break ...)

[void]

Currently:
0 = success
1 = error

I should expand the error codes more..

[stevelon]

Hmm. My Avast Anti-virus flags this new es.exe (Win32:Evo-gen (Suspected).

But I trust you void

[keithg1964]

Please include the command line program in the main release program. The file is so small, I would not think an need to have it as an optional install item since it compresses to about 40k.

Keith

[jg00d3]

I'm trying to use the ES command line app to search through my system for any files that match a list of filenames\wildcard that match files that are dropped by crypto malware.

I'm starting everything.exe -startup
then I run:
es.exe _secret_code.txt^|_ryp^|_Locky_recover_instructions.txt^|_Locky_recover_instructions.bmp^|_H_e_l_p_RECOVER_INSTRUCTIONS+*.txt^|_H_e_l_p_RECOVER_INSTRUCTIONS*.txt^|_H_e_l_p_RECOVER_INSTRUCTIONS*.png^|_H_e_l_p_RECOVER_INSTRUCTIONS*.html^|_how_recover.txt^|_how_recover*.txt^|_how_recover*.html^|_HELP_instructions.txt^|_HELP_instructions.bmp^|_help_instruct*.*^|_DECRYPT_INFO_*.html^|_crypt^|_*_HOWDO_text.html^|zzzzzzzzzzzzzzzzzyyy^|zycrypt.*^|YOUR_FILES_ARE_LOCKED.txt^|YOUR_FILES_ARE_ENCRYPTED.TXT^|YOUR_FILES_ARE_ENCRYPTED.HTML^|YOUR_FILES.url^|YOUR_FILES.HTML^|Your files encrypted by our friends !!! txt^|Your files are locked !.txt^|Your files are locked !!.txt^|Your files are locked !!!.txt^|Your files are locked !!!!.txt^|YOUGOTHACKED.TXT^|winclwp.jpg^|wie_zum_Wiederherstellen_von_Dateien.txt^|vault.txt^|vault.key^|vault.hta^|UNLOCK_FILES_INSTRUCTIONS.txt^|UNLOCK_FILES_INSTRUCTIONS.html^|UnblockFiles.vbs^|tox.html^|strongcrypt.bmp^|SIFRE_COZME_TALIMATI.html^|SHTODELATVAM.txt^|SECRETIDHERE.KEY^|SECRET.KEY^|Sarah_G@ausi.com___*^|RESTORE_FILES_*.txt^|RESTORE_FILES_*.*^|restore_files.txt^|restorefiles.txt^|recovery_key.txt^|RECOVERY_FILES.txt^|recovery_file.txt^|RECOVERY_FILE*.txt^|recoveryfile*.txt^|recovery+*.*^|recoverfile*.txt^|READ_THIS_TO_DECRYPT.html^|Read_this_file.txt^|READTHISNOW!!!.TXT^|README_RECOVER_FILES_*.txt^|README_RECOVER_FILES_*.png^|README_RECOVER_FILES_*.html^|README_HOW_TO_UNLOCK.TXT^|README_HOW_TO_UNLOCK.HTML^|readme_for_decrypt*.*^|README_DECRYPT_UMBRE_ID_*.txt^|README_DECRYPT_UMBRE_ID_*.jpg^|README_DECRYPT_HYRDA_ID_*.txt^|README_DECRYPT_HYDRA_ID_*.txt^|readme_decrypt*.*^|readme.hta^|ReadDecryptFilesHere.txt^|Read.txt^|READ ME FOR DECRYPT.txt^|Read Me (How Decrypt) !!!!.txt^|READ IF YOU WANT YOUR FILES BACK.html^|ransomed.html^|qwer2.html^|qwer.html^|pronk.txt^|Payment_Instructions.jpg^|paycrypt.bmp^|oor.*.*^

Currently there are 546 filename\wildcards in the list. I'm splitting the list and running it multiple times. everything.exe seems to crash w/ Everything IPS service not running when it hits this section of the filename\wildcards "RESTORE_FILES_*.txt^|RESTORE_FILES_*.*" if I remove those and run again, it chokes up on a similar combination. My guess is that it's because RESTORE_FILE_*.txt is also included in the RESTORE_FILES_*.* wildcard. Is there anyway to get it to skip duplicates instead of crashing?

I'm downloading this list of filename\wildcards from the web and it's updated often I'm trying to automate this so I don't have to manipulate the list manually.

Thanks!

[void]

Thanks for the bug report.

The issue should be fixed in Everything 796b:

Please include the command line program in the main release program.

I'll consider adding ES to the installer, and possibly the portable zip, thanks.

[jg00d3]

i'm having issues w/ running everything.exe -startup and es.exe in a script.

I'd really like to not have to run it in a batch file, but launching everything.exe -startup doesn't seem to exit or send so the my scripting environment sits and waits for the everything.exe process to stop before it proceeds.

So if I run the filescan0.txt (rename to bat) its successful the majority of the time unless everything.exe takes too long to scan the drive on startup. If I add some extra filename\wildcards to the es.exe search as in filescan1.txt (rename to bat) it hangs almost everytime.

what I'd like to do is start everything.exe -startup w/ my scripting application, then return back to the scripting application to run es.exe w/ the various searches that I want to perform. Once I've completed all the searches then run everything.exe -exit. That way I don't have to launch everything multiple times and have to scan the drive each time it opens.

Is there an everything command line option I am missing to get this to work properly or is there a way the es.exe can call everything.exe if it's not running?

It won't let me load .bat or .txt file attachments so here's the batch file lines:

filescan0.bat:
everything.exe -startup & timeout 1 & es.exe *.oplata@qq_com^|*.oor^|*.xlsx -export-txt % filename0.txt & everything.exe -exit


filescan1.bat
everything.exe -startup & timeout 1 & es.exe *.oplata@qq_com^|*.oor^|*.only-we_can-help_you^|*.one-we_can-help_you^|*.OMG!^|*.okean*^|*.odin^|*.odcodc^|*.nuclear55^|*.nochance^|*.neitrino^|*.nalog@qq_com^|*.micro^|*.magic_software_syndicate^|*.magic^|*.LOL!^|*.locky^|*.locked^|*.legion^|*.LeChiffre^|*.krypted^|*.kratos^|*.kraken^|*.kostya^|*.korrektor^|*.kkk^|*.kimcilware.locked^|*.kimcilware^|*.KEYZ.KEYH0LES^|*.KEYZ^|*.KEYHOLES^|*.KEYH0LES^|*.keybtc@inbox_com^|*.keybtc@inbox^|*.k0stya^|*.justbtcwillhelpyou^|*.iwanthelpuuu^|*.isis^|*.infected^|*.hydracrypt_ID_*^|*.hydracrypt_ID*^|*.How_To_Get_Back.txt^|*.How_To_Decrypt.txt^|*.herbst^|*.helpdecrypt@ukr_net^|*.helpdecrypt@ukr.net^|*.helpdecrypt@ukr*.net^|*.heisenberg^|*.hb15^|*.ha3^|*.h3ll^|*.gws.porno^|*.gws^|*.gruzin@qq_com^|*.good^|*.globe^|*.fun^|*.fuckyourdata^|*.fucked^|*.fuck^|*.frtrss^|*.fileiscryptedhard^|*.fantom^|*.ezz^|*.exx^|*.exotic^|*.evil^|*.epic^|*.enigma^|*.encryptedRSA^|*.encryptedped^|*.encryptedAES^|*.encrypted.locked^|*.Encrypted^|*.encrypt^|*.encoderpass^|*.EnCiPhErEd^|*.encedRSA^|*.eclr^|*.ecc^|*.dyatel@qq_com^|*.dxxd^|*.domino^|*.ded^|*.dCrypt^|*.da_vinci_code^|*.darkness^|*.czvxce^|*.CTBL2^|*.ctbl^|*.CTB2^|*.CrySiS^|*.crypz^|*.cryptz^|*.crypttt^|*.cryptowall^|*.CryptoTorLocker2015!^|*.cryptotorlocker*^|*.cryptolocker^|*.crypto^|*.xlsx -export-txt % filename1.txt & everything.exe -exit


Jg00d3

[keithg1964]

(Post contained incorrect information, and that info was deleted.)

KG

[void]

http://www.voidtools.com/es-1.1.0.3a.zip

Added the command line option: -timeout <milliseconds>
Timeout after the specified number of milliseconds to wait for the Everything database to load before sending a query.

For example, myES.bat :

Code: Select all

START Everything.exe -startup
es.exe -timeout 10000 <es search text...>
Everything.exe -exit

Added return codes:

Code: Select all

ES_ERROR_SUCCESS                    0 // no known error, search successful.
ES_ERROR_REGISTER_WINDOW_CLASS      1 // failed to register window class
ES_ERROR_CREATE_WINDOW              2 // failed to create listening window.
ES_ERROR_OUT_OF_MEMORY              3 // out of memory
ES_ERROR_EXPECTED_SWITCH_PARAMETER  4 // expected an additional command line option with the specified switch
ES_ERROR_CREATE_FILE                5 // failed to create export output file
ES_ERROR_UNKNOWN_SWITCH             6 // unknown switch.
ES_ERROR_SEND_MESSAGE               7 // failed to send Everything IPC a query.
ES_ERROR_IPC                        8 // NO Everything IPC window - make sure the Everything client is running.

ES does not use the Everything service, ES only communicates with the Everything search client.

[jg00d3]

Very cool! thanks!

But I'm seeing something strange. I come up with this batch file which splits the files to check into multiple es.exe commands. All should find something because of the last wildcard and export the results to a txt file, but only filename1.txt contains anything. I'm echoing %errorlevel% and the errorlevel is always 0.


@Echo Off
Start %windir%\packages\everything\everything.exe -startup
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe _secret_code.txt^|_ryp^|_Locky_recover_instructions.txt^|_Locky_recover_instructions.bmp^|_H_e_l_p_RECOVER_INSTRUCTIONS+*.txt^|_H_e_l_p_RECOVER_INSTRUCTIONS*.txt^|_H_e_l_p_RECOVER_INSTRUCTIONS*.png^|_H_e_l_p_RECOVER_INSTRUCTIONS*.html^|_how_recover.txt^|_how_recover*.txt^|_how_recover*.html^|_HELP_instructions.txt^|_HELP_instructions.bmp^|_help_instruct*.*^|_DECRYPT_INFO_*.html^|_crypt^|_*_HOWDO_text.html^|zzzzzzzzzzzzzzzzzyyy^|zycrypt.*^|YOUR_FILES_ARE_LOCKED.txt^|YOUR_FILES_ARE_ENCRYPTED.TXT^|YOUR_FILES_ARE_ENCRYPTED.HTML^|YOUR_FILES.url^|YOUR_FILES.HTML^|Your files encrypted by our friends !!! txt^|Your files are locked !.txt^|Your files are locked !!.txt^|Your files are locked !!!.txt^|Your files are locked !!!!.txt^|YOUGOTHACKED.TXT^|winclwp.jpg^|wie_zum_Wiederherstellen_von_Dateien.txt^|vault.txt^|vault.key^|vault.hta^|UNLOCK_FILES_INSTRUCTIONS.txt^|UNLOCK_FILES_INSTRUCTIONS.html^|UnblockFiles.vbs^|tox.html^|strongcrypt.bmp^|SIFRE_COZME_TALIMATI.html^|SHTODELATVAM.txt^|SECRETIDHERE.KEY^|SECRET.KEY^|Sarah_G@ausi.com___*^|RESTORE_FILES_*.txt^|RESTORE_FILES_*.*^|*.docx -dm -sort dm -export-txt %windir%\packages\everything\filename5.txt
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe restore_files.txt^|restorefiles.txt^|recovery_key.txt^|RECOVERY_FILES.txt^|recovery_file.txt^|RECOVERY_FILE*.txt^|recoveryfile*.txt^|recovery+*.*^|recoverfile*.txt^|READ_THIS_TO_DECRYPT.html^|Read_this_file.txt^|READTHISNOW!!!.TXT^|README_RECOVER_FILES_*.txt^|README_RECOVER_FILES_*.png^|README_RECOVER_FILES_*.html^|README_HOW_TO_UNLOCK.TXT^|README_HOW_TO_UNLOCK.HTML^|readme_for_decrypt*.*^|README_DECRYPT_UMBRE_ID_*.txt^|README_DECRYPT_UMBRE_ID_*.jpg^|README_DECRYPT_HYRDA_ID_*.txt^|README_DECRYPT_HYDRA_ID_*.txt^|readme_decrypt*.*^|readme.hta^|ReadDecryptFilesHere.txt^|Read.txt^|READ ME FOR DECRYPT.txt^|Read Me (How Decrypt) !!!!.txt^|READ IF YOU WANT YOUR FILES BACK.html^|ransomed.html^|qwer2.html^|qwer.html^|pronk.txt^|Payment_Instructions.jpg^|paycrypt.bmp^|oor.*.*^|oor.*^|oor*.^|ONTSLEUTELINGS_INSTRUCTIES.html^|OKSOWATHAPPENDTOYOURFILES.TXT^|message.txt^|MENSAGEM.txt^|maxcrypt.bmp^|last_chance.txt^|KryptoLocker_README.txt^|ISTRUZIONI_DECRITTAZIONE.html^|INSTRUCTIONS_DE_DECRYPTAGE.html^|Instructionaga.txt^|INSTRUCCIONES_DESCIFRADO.TXT^|INSTRUCCIONES_DESCIFRADO.html^|INSTALL_TOR.URL^|install_tor*.*^|IMPORTANT.README^|IMPORTANT READ ME.txt^|IHAVEYOURSECRET.KEY^|IAMREADYTOPAY.TXT^|HOW_TO_UNLOCK_FILES_README_*.txt^|how_to_unlock*.*^|HOW_TO_RESTORE_FILES.html^|How_to_restore_files.hta^|How_To_Recover_Files.txt^|how_to_recover*.*^|How_to_decrypt_your_files.jpg^|HOW_TO_DECRYPT_FILES.TXT^|HOW_TO_DECRYPT.HTML^|how_to_decrypt*.*^|how_recover*.*^|HOW_DECRYPT.URL^|HOW_DECRYPT.TXT^|HOW_DECRYPT.HTML^|how_decrypt.gif^|Howto_Restore_FILES.TXT^|Howto_RESTORE_FILES.html^|howto_restore*.*^|HOWTO_RECOVER_FILES_*.*^|howto_recover_file.txt^|HowtoRESTORE_FILES.txt^|howtodecryptaesfiles.txt^|howtodecrypt*.*^|howrecover+*.txt^|HowDecrypt.txt^|HowDecrypt.gif^|HOW-TO-DECRYPT-FILES.HTML^|how to get data.txt^|how to decrypt*.*^|How to decrypt your data.txt^|How to decrypt LeChiffre files.html^|HOW TO DECRYPT FILES.txt^|HOW TO DECRYPT FILES.HTML^|how to decrypt aes files.lnk^|HELP_YOUR_FILES.TXT^|HELP_YOUR_FILES.PNG^|HELP_YOUR_FILES.html^|help_your_file*.*^|HELP_YOURFILES.HTML^|HELP_TO_SAVE_FILES.txt^|HELP_TO_SAVE_FILES.bmp^|HELP_TO_DECRYPT_YOUR_FILES.txt^|HELP_RESTORE_FILES_*.*^|HELP_RESTORE_FILES.txt^|*.xlsx -dm -sort dm -export-txt %windir%\packages\everything\filename4.txt
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe help_restore*.*^|help_recover_instructions+*.txt^|help_recover_instructions*.txt^|help_recover_instructions*.html^|help_recover_instructions*.bmp^|HELP_RECOVER_FILES.txt^|help_recover*.*^|help_instructions.*^|help_file_*.*^|help_decrypt_your_files.html^|HELP_DECRYPT.URL^|Help_Decrypt.txt^|HELP_DECRYPT.PNG^|HELP_DECRYPT.lnk^|HELP_DECRYPT.HTML^|help_decrypt*.*^|HELPDECYPRT_YOUR_FILES.HTML^|HELPDECRYPT.TXT^|help-file-decrypt.enc^|Help Decrypt.html^|HELLOTHERE.TXT^|Hacked_Read_me_to_decrypt_files.html^|GetYouFiles.txt^|FILES_BACK.txt^|FILESAREGONE.TXT^|File Decrypt Help.html^|exit.hhr.obleep^|ENTSCHLUSSELN_HINWEISE.html^|enigma_encr.txt^|enigma.hta^|enc_files.txt^|enc_files* ^|encryptor_raas_readme_liesmich.txt^|encryptor_raas_readme_liesmich* ^|email-salazar_slytherin10@yahoo.com.ver-*.id-*-*.randomname-*^|de_crypt_readme.txt^|de_crypt_readme.html^|de_crypt_readme.bmp^|de_crypt_readme.*^|DESIFROVANI_POKYNY.html^|default432643264.jpg^|default32643264.bmp^|decypt_your_files.html^|DECRYPT_YOUR_FILES.txt^|DECRYPT_YOUR_FILES.HTML^|DECRYPT_ReadMe1.TXT^|DECRYPT_Readme.TXT.ReadMe^|DECRYPT_ReadMe.TXT^|DECRYPT_ReadMe* ^|DECRYPT_INSTRUCTIONS.TXT^|DECRYPT_INSTRUCTIONS.html^|DECRYPT_INSTRUCTION.URL^|DECRYPT_INSTRUCTION.TXT^|DECRYPT_INSTRUCTION.HTML^|decrypt_instruct*.*^|DECRYPTION_HOWTO.Notepad^|DecryptAllFiles.txt^|DecryptAllFiles*.txt^|DecryptAllFiles* ^|decrypt-instruct*.*^|Decrypt All Files *.bmp^|cryptolocker.*^|cryptinfo.txt^|crjoker.html^|confirmation.key^|Como descriptografar seus arquivos.txt^|Comment du00e9bloquer mes fichiers.txt^|Coin.Locker.txt^|BUYUNLOCKCODE.txt^|BUYUNLOCKCODE^|BitCryptorFileList.txt^|ATTENTION!!!.txt^|AllFilesAreLocked*.bmp^|About_Files.txt^|4-14-2016-INFECTION.TXT^|.~^|*_WHAT_is.html^|*_ryp^|*_recover_*.*^|*_nullbyte*^|*_HOWDO_text.html^|*_HOWDO_text.bmp^|*_HELP_instructions.html^|*_help_instruct*.* ^|*warning-!!*.*^|*want your files back.*^|*ukr.net*^|*restore_fi*.*^|*recover}-*.*^|*recover_instruction*.*^|*rec0ver*.*^|*qq_com*^|*maestro@pizzacrypts.info^|*keemail.me*^|*install_tor*.*^|*how_to_recover*.*^|*help_restore*.*^|*gmail*.crypt^|*files_are_encrypted.*^|*DECRYPT_INSTRUCTION*^|*.txt -dm -sort dm -export-txt %windir%\packages\everything\filename3.txt
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe *decryptmyfiles*.*^|*decrypt your file*.*^|*decrypt my file*.*^|*decipher*^|*cpyt*^|*cerber2^|*@india.com*^|*@gmail_com_*^|*.~HL*^|*.{CRYPTENDBLACKDC}^|*._ryp^|*._AiraCropEncrypted!^|*.zzz^|*.zypto*^|*.zyklon^|*.Zimbra^|*.zepto^|*.zcrypt^|*.zc3791^|*.Z81928819^|*.YTBL^|*.Yakes^|*.xyz^|*.xxx^|*.xtbl^|*.xrtn^|*.XRNT^|*.xort^|*.xorist^|*.xcrypt^|*.windows10^|*.Where_my_files.txt^|*.wflx^|*.vvv^|*.vscrypt^|*.venusp^|*.Venusf^|*.vault^|*.usr0^|*.unbrecrypt_ID_*^|*.unavailable^|*.uc554ud638ud654ub428^|*.u5df2u52a0u5bc6^|*.tzu^|*.ttt^|*.trun^|*.troyancoder@qq_com^|*.toxcrypt^|*.thor^|*.szf^|*.surprise^|*.SUPERCRYPT^|*.sshxkej^|*.sport^|*.Silent^|*.shit^|*.shino^|*.SecureCrypted^|*.SecureCrypte^|*.scl^|*.sanction^|*.ryp^|*.rtyrtyrty^|*.RSplited^|*.RSNSlocked^|*.RRK^|*.rokku^|*.rnsmwr^|*.rip^|*.remind^|*.relock@qq_com^|*.rekt^|*.realfs0ciety*^|*.Read_Me.Txt^|*.RDM^|*.razy^|*.raid10^|*.RADAMANT^|*.RAD^|*.R5A^|*.R4A^|*.R16M01D05^|*.pzdc^|*.purge^|*.POSHKODER^|*.pornoransom^|*.porno.pornoransom^|*.porno^|*.PoAr2w^|*.plauge17^|*.pizda@qq_com^|*.pdcr^|*.payransom^|*.paymst^|*.payms^|*.payfornature@india.com.crypted^|*.paybtcs^|*.padcrypt^|*.p5tkjw^|*.oshit^|*.bin -dm -sort dm -export-txt %windir%\packages\everything\filename2.txt
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe *.oplata@qq_com^|*.oor^|*.only-we_can-help_you^|*.one-we_can-help_you^|*.OMG!^|*.okean*^|*.odin^|*.odcodc^|*.nuclear55^|*.nochance^|*.neitrino^|*.nalog@qq_com^|*.micro^|*.magic_software_syndicate^|*.magic^|*.LOL!^|*.locky^|*.locked^|*.legion^|*.LeChiffre^|*.krypted^|*.kratos^|*.kraken^|*.kostya^|*.korrektor^|*.kkk^|*.kimcilware.locked^|*.kimcilware^|*.KEYZ.KEYH0LES^|*.KEYZ^|*.KEYHOLES^|*.KEYH0LES^|*.keybtc@inbox_com^|*.keybtc@inbox^|*.k0stya^|*.justbtcwillhelpyou^|*.iwanthelpuuu^|*.isis^|*.infected^|*.hydracrypt_ID_*^|*.hydracrypt_ID*^|*.How_To_Get_Back.txt^|*.How_To_Decrypt.txt^|*.herbst^|*.helpdecrypt@ukr_net^|*.helpdecrypt@ukr.net^|*.helpdecrypt@ukr*.net^|*.heisenberg^|*.hb15^|*.ha3^|*.h3ll^|*.gws.porno^|*.gws^|*.gruzin@qq_com^|*.good^|*.globe^|*.fun^|*.fuckyourdata^|*.fucked^|*.fuck^|*.frtrss^|*.fileiscryptedhard^|*.fantom^|*.ezz^|*.exx^|*.exotic^|*.evil^|*.epic^|*.enigma^|*.encryptedRSA^|*.encryptedped^|*.encryptedAES^|*.encrypted.locked^|*.Encrypted^|*.encrypt^|*.encoderpass^|*.EnCiPhErEd^|*.encedRSA^|*.eclr^|*.ecc^|*.dyatel@qq_com^|*.dxxd^|*.domino^|*.ded^|*.dCrypt^|*.da_vinci_code^|*.darkness^|*.czvxce^|*.CTBL2^|*.ctbl^|*.CTB2^|*.CrySiS^|*.crypz^|*.cryptz^|*.crypttt^|*.cryptowall^|*.CryptoTorLocker2015!^|*.cryptotorlocker*^|*.cryptolocker^|*.crypto^|*.csv -dm -sort dm -export-txt %windir%\packages\everything\filename1.txt
Echo %errorlevel%
pause
%windir%\packages\everything\es.exe *.crypted^|*.crypt38^|*.crypt1^|*.crypt^|*.cryp1^|*.cry^|*.CRRRT^|*.crptrgr^|*.crjoker^|*.crjocker^|*.cripttt^|*.criptokod^|*.criptiko^|*.crinf^|*.crime^|*.cRh8^|*.crashed^|*.coverton^|*.Contact_Here_To_Recover_Your_Files.txt^|*.comrade^|*.coded^|*.code^|*.clf^|*.chifrator@qq_com^|*.cerber3^|*.cerber2^|*.cerber^|*.CCCRRRPPP^|*.ccc^|*.cbf^|*.cawwcca^|*.canihelpyou^|*.C0rp0r@c@0Xr@^|*.btcbtcbtc^|*.btc.kkk.fun.gws^|*.btc-help-you^|*.btc^|*.breaking_bad^|*.breaking bad^|*.bloccato^|*.blocatto^|*.bloc^|*.bleepYourFiles^|*.bleep^|*.bitstak^|*.better_call_saul^|*.bart.zip^|*.bart^|*.b5c6^|*.axx^|*.asdasdasd^|*.ap19^|*.amba^|*.Alcatraz^|*.akaibvn^|*.aga^|*.AFD^|*.AES256^|*.abc^|*.aaa^|*.a5zfn^|*.a19^|*.96e2^|*.8lock8^|*.7z.encrypted^|*.7h9r^|*.777^|*.73i87A^|*.6FKR8d^|*.31392E30362E32303136_*^|*.31342E30362E32303136*^|*.1txt^|*.1999^|*.0x0^|*.*obleep^|*.*nochance^|*.*locked^|*.*kraken^|*.*kb15^|*.*exx^|*.*darkness^|*.*crypto^|*.*cry^|*-recover-*.*^|*+recover+*.*^|*!recover!*.*^|*!DMAlock* ^|$RECYCLE.BIN.{*-*-*-*}^|# DECRYPT MY FILES #.vbs^|# DECRYPT MY FILES #.txt^|# DECRYPT MY FILES #.html^|!Where_are_my_files!.html^|!satana!.txt^|!Recovery_*.txt^|!readme.*^|!DMALOCK3.0*^|!!!README!!!*.rtf^|!!!-WARNING-!!!.txt^|!!!-WARNING-!!!.html^|*.reg -dm -sort dm -export-txt %windir%\packages\everything\filename0.txt
Echo %errorlevel%
pause
%windir%\packages\everything\everything.exe -exit


Also, anyway to get the -dm -sort dm to come out in the export-txt?

Thanks!

jg00d3

[froggie]

You need double quotes for names with spaces - filename1 input parameters work because there are no spaces in its names

[void]

Also, please use double quotes to escape search terms starting with !
Search terms starting with ! means NOT.
|"!DMALOCK3.0*"^|

[jg00d3]

Awesome, thanks so much!

Is there a way to get the dm and other settings to export when using -export-txt?

jg00d3

[void]

Currently, -export-txt only exports the full path and file name.

Please try using > output redirection to save all columns to an output file, for example:
es.exe _secret_code.txt^|_ryp^|_Locky_recover_instructions.txt^|_Locky_recover_instructions.bmp^|_H_e_l_p_RECOVER_INSTRUCTIONS+*.txt -dm -sort dm > %windir%\packages\everything\filename5.txt

[void]

http://www.voidtools.com/es-1.1.0.4a.zip

• Improved -pause/-more
• Added -set-run-count <filename> <run-count>
• Added -inc-run-count <filename>
• Added -get-run-count <filename>
• Columns can now be re-ordered.
• Improved column layout.

-pause/-more now shows a scrollable window, not sure if this is really an improvement, let me know what you think..
Q or Esc = Quit
Arrow Keys, Page Up, Page Down, Home and End = Scroll.
Enter = Scroll one line and exit on last line
Space = Scroll one page and exit on last page.

-set-run-count, -inc-run-count and -get-run-count requires Everything 1.4.1.802b or later.

[therube]

-pause/-more now shows a scrollable window, not sure if this is really an improvement, let me know what you think..

Really getting fancy now .
w - wrap
n - number
/ - search
h - help
r - write

Default "prompt", ":".
If you want help, :h.
Save the stream to a file, :r c:\tmp\out.txt

Maybe a configurable prompt, so the "current" (bottom-most) line, total count of results.
[23/72]:

And at that point, you're much like Less.

I like it.
Certainly better.

If the number of returned results is the same as the window size (size-1), number of rows, its not quit obvious that there are no more results, so up/down, home/end... seem ineffectual. (Not a big deal, but.)

[keithg1964]

Feature Request for ES.exe If no files found do not show the following line:

"ESC=Quit; Up,Down,Left,Right,Page Up,Page Down,Home,End=Scroll"

just go straight back to command line with an short message stating nothing found.

Thanks for great software.

[void]

http://www.voidtools.com/es-1.1.0.5a.zip

• Added full attribute searching: /a[RHSDAVNTPLCOIE]
• -pause/-more ignored if all results are visible.
• Added -sort-size, -sort-size-ascending and -sort-size-descending style sorts command line options.

And at that point, you're much like Less.

Added to my TODO list, thanks.

[nod5]

The new ES is great! Dramatic speed improvement when saving whole drive .efu files from command line compared to old method using everything.exe command line.

I made a small helper script for saving individual datestamped .efu files for multiple drives in one go, see end of first post here http://www.donationcoder.com/forum/inde ... #msg401607 .

[void]

http://www.voidtools.com/es-1.1.0.6a.zip

• added -csv, -efu, -txt, -m3u and -m3u8 to set the display format (does not export)
• added -export-m3u and -export-m3u8.

[regios]

This worked in ES 1.1.0.4a

Code: Select all

es.exe "C:\" -efu "D:\a.efu"

But it does not work for me in ES 1.1.0.6a or 1.1.0.8. In those newer versions the cmd prompt only prints the string "Filename,Size,Date Modified,Date Created,Attributes" and no .efu file is saved.

Edit: never mind, the syntax had changed. This works in 1.1.0.8

Code: Select all

es.exe -export-efu "D:\a.efu" "C:\"

[therube]

ES 1.1.0.8
Everything 1.4.1.892

Code: Select all

es.exe  -path .  -pause  .ini

Code: Select all

C:\WLIB\LeechFTP\LEECHTMP\ip19655837.ahodd.ini\key=yyIDkiKLgryN0-lt14zUDw,end=1513485194\state=RfoI\buffer=73380007338000,7265.5\speed=97545\reftag=058555760\14\121\9\31818479\c6\300000\373000\373690\testing123.ini

-pause does not allow LONG lines to wrap (or to scroll horizontally to the end).
-pause truncates long line output (to some extent, with viewport [window] width playing a factor, so if you resize, you may see more or less, but never in full).

The file is found, the line is displayed, but it is truncated.
It should be ~214 characters.
What actually ends up displaying may vary, perhaps between 92 & 136 characters (perhaps larger?), with the display width playing into what you get.


(In my case, I started the ES search from within the /ip19655837.ahodd.ini/ directory.)

[NotNull]

ES 1.1.0.8
Everything 1.4.1.892

-pause does not allow LONG lines to wrap (or to scroll horizontally to the end).
-pause truncates long line output (to some extent, with viewport [window] width playing a factor, so if you resize, you may see more or less, but never in full).

ES.exe has it's own, build-in, scrolling mechanism that is independent of the console host implementation.
Use cursor key right to see the missing part of the line. That should do the trick according to the -pause instructions at the bottom of your results:

ESC=Quit; Up,Down,Left,Right,Page Up,Page Down,Home,End=Scroll

BTW1: I usually resize the if I expect long lines (with WMIC, for example) :

Code: Select all

mode 300,40

That gives you a width of 300 characters and a height of 40.
Note that this is semi-independent of your "viewport"

BTW2: In this case your Windows (sub)version is relevant too. Recently Windows had some major changes (improvements in my opinion) in Console Host (conhost.exe is responsible for input/output to and from console applications like CMD.exe andnPowershell.exe)

[void]

-pause does not allow LONG lines to wrap

This is by design.

(or to scroll horizontally to the end).

Left Arrow Key / Right Arrow Key should scroll the view.
Do you see any scrolling? Does your console window have scroll bars?

ES -pause uses the current viewport rather than the size of the console.

So you might see this truncation when you have a console size of say 100x40, with a viewport of 80x40 (you should see a horizontal scroll bar at the bottom of the console window) If you use the scrollbar you will see that -pause only scrolls the initial viewport.

Resizing the console window in -pause also causes issues.

I've tested on Windows 7 and 10 and ES -pause is scrolling fine for really really long filenames (full path and filename length > 260).
Please make sure your console does not have scroll bars avoid resizing the console inside ES -pause.

I've made a note in my ES TODO list to improve -pause when the console window is resized.

[nod5]

Speed comparison for three ways to export a disk index as an .efu file
- The tests were done on a SSD with ~650K files and folders
- Using ES 1.1.0.8 and Everything 1.4.1.877 (x64) in Windows 10

Test 1. ES command line
es.exe -export-efu C:\es_test.efu C:\
Result: ~35 seconds 90MB output file

(Same time with -sort-paths command added.)
(Same time when running es as admin.)

Test 2. Everything.exe command line
Everything.exe -create-file-list C:\ev_test.efu C:\
Result: ~15 seconds 93MB output file
(Note: same time running as non-admin and admin, but the non-admin created .efu omitted ~20K files)

Test 3: Everything GUI menu
Everything GUI: first search C:\ and then go to menu > file > export
~2 seconds to export 78MB output file

Differences in output
From a quick glance only test 3 .efu differs in these ways
- test 3: no file attribute data (only 16 and 0)
- test 3: no date-created data (only date-modified)
- test 3: folders have no size value (test 1 and 2: folders have size value 0)

The ES method used to be about as fast as the GUI menu export back in November 2016. I've since then been using the ES method as part of some automatic backup scripts. I don't know at what version of ES/Everything the speed decreased.

Questions
- Can others reproduce that ES has gotten much slower at .efu export?
- Is there currently some command line method (with ES or Everything) to get the same data at the same speed as in the Everything GUI menu method from test 3? Assuming Everything is already running in the background on the system. Some option or parameter I'm missing? I did try the ES .csv export option with different settings but got similar results as in test 1.

[therube]

Guessing...

Everything GUI is working from RAM, is not exporting all the detail that ES is.
ES is exporting detail that is not in RAM, not in .db, has to go to disk to gather those details.


As far as ES time, I'm seeing no difference between versions.
(There is a version from 2014, but it is far more basic, has no -export-efu...)
All later versions are running (the particular search I ran), on average, in .26 sec.

"Caching" can have a large impact on time. So if something "interferes" with what has been cached, results will be skewed.
(I'm not quite sure what I'm seeing in this regard. Sometimes extraneous activity affects things, sometimes killing a long running ES command followed by running the same again causes it to complete much quicker...)


Also note, Differences between the exported formats.


go.bat:

Code: Select all

upct.exe  .\es.exe -export-efu out.efu smith
pause
exit

Code: Select all

C:\DEV\Locate\ES>upct.exe  .\es.exe -export-efu out.efu smith
┌───────────────────────────────────────────────────────────────────────┐
│ Ultra Precision Command Timer 1.6 - Freeware (C) 1993 by Erik de Neve │
└───────────────────────────────────────────────────────────────────────┘
   Full Command Line: .\es.exe  -export-efu out.efu smith
   Starting: .\es.exe


 Command executed:  .\ES.EXE  -export-efu out.efu smith
   Parameters:   -export-efu out.efu smith
 Raw total count: 315822 cycles on the 8253/4 programmable interval timer.
 Elapsed time:      264688.9 microseconds =  0.2647 seconds.

C:\DEV\Locate\ES>pause
Press any key to continue . . .

Code: Select all

┌───────────────────────────────────────────────────────────────────────┐
│ Ultra Precision Command Timer 1.6 - Freeware (C) 1993 by Erik de Neve │
└───────────────────────────────────────────────────────────────────────┘

   Example:   C:\> UPCT  [COMMAND_SEQUENCE]
  ┌──────────────────────────────┘
  └─ Meaning: Any command plus parameters you would normally
      enter at the DOS command prompt, including redirection.

[NotNull]

Speed comparison for three ways to export a disk index as an .efu file
- Can others reproduce that ES has gotten much slower at .efu export?

I'm quite new to Everything, so no historic comparison possible, but ..

Here you go
- My tests were done on a very slow HDD with ~280K files and folders; old Celeron something processor
- Using ES 1.1.0.8 and Everything 1.4.1.885 (x64) in Windows 10

Run in PowerShell; after each test I did quit Everything and started it again.

Code: Select all

PS C:\Tools\Everything> (Measure-command { .\es.exe -export-efu test1.efu C:\ }).TotalSeconds
56,6275392

PS C:\Tools\Everything> (Measure-command { .\Everything.exe -create-file-list test2.efu C:\ }).TotalSeconds
0,1225252

There is another , important difference between the outputs: they are sorted different!
That means that ES.exe asks the GUI version (Everything search client) to produce all results from it's database. These results have to be sorted and "massaged". That takes quite some (CPU) effort and thus time.

To speed up the sorting, you normally use indexes. For this test I just enabled all indexes, including it's fast sort option (under Menu:Tools > Options > General). Feel free to research which index has the most influence on the speed (and please report the resulst here )


This is the new testresult for Test 1:

Code: Select all

PS C:\Tools\Everything> (Measure-command { .\es.exe -export-efu test1.efu C:\ }).TotalSeconds
8,1462335

Also: done in ~8 seconds instead of ~56.

My guess is that ES.exe and Everything.exe are no longer "aligned" : ES.exe asks for layout 1; Everything.exe defaults to layout 2. Hence the extra time.


I don't know how you got different results running Ethe queries as a normal user vs running as an administrator (assuming you had the Everything service running). That should not be possible. Both talk to the (same) database, not the system.

EDIT: Upon rereading: This s not exactly what I wanted to say. (some reasoning is wrong). Will rewrite it later.

[nod5]

therube & notnull: thank you for also testing.

Everything GUI is working from RAM, is not exporting all the detail that ES is.
ES is exporting detail that is not in RAM, not in .db, has to go to disk to gather those details.

Plausible.

There is another , important difference between the outputs: they are sorted different!

Yes but using ES with the -sort-paths parameter (version of test 1) sorts the efu like in Everything exports (test 2/3). That parameter doesn't affect the speed of ES.

[NotNull]

Yes but using ES with the -sort-paths parameter (version of test 1) sorts the efu like in Everything exports (test 2/3). That parameter doesn't affect the speed of ES.

I was already aware of that (hence my "This s not exactly what I wanted to say. (some reasoning is wrong). Will rewrite it later."), but I was too tired to fix it at that moment. Sorry for confusion.

When I ran your ES quuery, it took 50+ seconds. All of that time CPU usage was somewhere between 60% and 70%. The only explanation I could come up with at that time was the sorting (as I said: very tired ..)

Now I think it is caused by all the extra info that isn't (by default) in the database, like attributes. Index those and you will get results much quicker.

[nod5]

Now I think it is caused by all the extra info that isn't (by default) in the database, like attributes. Index those and you will get results much quicker.

Possible. I'll test that later. Though I guess that would lead to Everything itself indexing the disks slower.

I hope the Everything dev, void, comments and straightens this out.