Keep offline volumes in the index

If you are experiencing problems with "Everything", post here for assistance.
void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Keep offline volumes in the index

Postby void » Mon Jul 07, 2014 6:43 am

To keep offline volumes in Everything 1.4 or later:
  • In Everything, from the Tools menu, click Options.
  • Click the NTFS tab.
  • Uncheck Automatically remove offline volumes.
  • Click OK.

When enabled, keep offline volumes will:
  • Allow offline volumes to stay in the index.
  • Not rebuild the database when the volume goes offline.
  • Automatically update the index when the volume comes back online with out doing a full rebuild.
  • Rebuild the database if a volume has changed too much.
  • Detect changes to the volume while it is online.
  • Offline indexes will be lost when the index is rebuilt. All volumes must be online when Everything builds its index.

Enable keep offline volumes in Everything 1.3
To enable keep missing volumes:
  • Exit Everything
  • Edit your Everything.ini
  • add the following line to the end of the Everything.ini:

    Code: Select all

    keep_missing_indexes=1
  • Save changes and restart Everything.

Requirements:
Requires Everything 1.3.4.672b or later

galneon
Posts: 26
Joined: Sun Jun 02, 2013 1:36 am

Re: Keep offline volumes in the index

Postby galneon » Mon Jul 07, 2014 4:15 pm

A great feature I've been waiting for. Is it tied strictly to volume letter? Curious to know under what conditions an index may be lost to a similarly named volume of another drive.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Tue Jul 08, 2014 2:02 am

Is it tied strictly to volume letter?


Everything identifies volumes by their volume GUID and path.
If either change Everything must rebuild its database.

If you have volumes that share the same path, Everything will still rebuild the database if the online volume GUID differs to the one indexed.
I hope to improve this in a future release.

galneon
Posts: 26
Joined: Sun Jun 02, 2013 1:36 am

Re: Keep offline volumes in the index

Postby galneon » Thu Jul 10, 2014 11:37 pm

Hmm I think I've hit an issue different from that limitation. This has happened to me twice now on 672b: When I disconnect a drive, Everything turns empty (0 objects) until I close and restart it, then it rebuilds the database. That was with keep_missing_volumes=1, but after reloading Everything and checking the ini, the line is gone (independent of the drive disconnect issue--the line disappears every time I run the application). I've tried several times now and I can't seem to get the option to stick. I'm now on 676b and same story.

sheppaul
Posts: 24
Joined: Sat Jan 12, 2013 5:58 pm

Re: Keep offline volumes in the index

Postby sheppaul » Thu Jul 10, 2014 11:53 pm

Wow, I've never guessed if it is possible to do like this.. What an amazing feature!

galneon wrote:Hmm I think I've hit an issue different from that limitation. This has happened to me twice now on 672b: When I disconnect a drive, Everything turns empty (0 objects) until I close and restart it, then it rebuilds the database. That was with keep_missing_volumes=1, but after reloading Everything and checking the ini, the line is gone (independent of the drive disconnect issue--the line disappears every time I run the application).


I have a same issue. By the way, I really love this feature!!!

ps. Is it possible to distinguish between online and offline files visually?

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Fri Jul 11, 2014 2:20 am

Hmm I think I've hit an issue different from that limitation. This has happened to me twice now on 672b: When I disconnect a drive, Everything turns empty (0 objects) until I close and restart it, then it rebuilds the database. That was with keep_missing_volumes=1, but after reloading Everything and checking the ini, the line is gone (independent of the drive disconnect issue--the line disappears every time I run the application).


Do you have store settings and data in %APPDATA%\Everything option disabled in Tools -> Options -> General?

Are you changing the %APPDATA%\Everything\Everything.ini?

Do you have other installations of Everything installed? Check if the old Everything 1.2 is still running on start-up, this could cause the Everything.ini to partially reset.

What is displayed in the debug console when disconnecting a drive and Everything builds?

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Fri Jul 11, 2014 10:16 am

Is it possible to distinguish between online and offline files visually?
I will look into making offline files grayed.

galneon
Posts: 26
Joined: Sun Jun 02, 2013 1:36 am

Re: Keep offline volumes in the index

Postby galneon » Fri Jul 11, 2014 4:30 pm

I do have that setting disabled as I have a portable Everything (on a drive that is always live). I'm editing Everything.ini within my Everything directory, which is also where the db is (Database location path correctly points to it). This is definitely the only instance of Everything running or even present. It's the x64 build.

http://pastebin.com/zhFY4P73

I connected drive O: with Everything not running, ran Everything making sure keep_missing_volumes=1 was in the .ini, then disconnected O: once Everything was idle. Nothing happened. I connected O: again, this time while Everything was running (this is how I would normally connect a drive), and the db was rebuilt. I disconnected O:, and the DB was rebuilt again, this time excluding the now-offline O:. I then checked Everything.ini to find keep_missing_volumes=1 had disappeared entirely. This is consistent with the behavior I experienced twice before.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Sat Jul 12, 2014 2:12 am

Thanks for the debug output.

GetOverlappedResult failed 21 (O:) retrying in 30 seconds...
open volume v
CreateFileW(): GetLastError(): 2: Failed to open volume v
opened -1 0.000952
monitor thread exit


From the debug output I can see that the monitor thread exits when the volume fails to open, causing the db to rebuild.
This should never happen if keep_missing_indexes is set to 1.

Did you completely exit Everything before adding this setting to the Everything.ini?

galneon
Posts: 26
Joined: Sun Jun 02, 2013 1:36 am

Re: Keep offline volumes in the index

Postby galneon » Sun Jul 13, 2014 7:24 am

Yes. I've been as clinical as I can possibly be I'm afraid. I'll try an entirely virgin setup sometime soon with strictly default settings, with the exception of keep_missing_volumes=1 of course. This is the last remaining feature I really need in order to replace Cathy entirely.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Tue Jul 15, 2014 10:28 am

Updated, I had the wrong ini setting name above, it was keep_missing_volumes, but it should be:

Code: Select all

keep_missing_indexes=1


Sorry for the inconvenience.

galneon
Posts: 26
Joined: Sun Jun 02, 2013 1:36 am

Re: Keep offline volumes in the index

Postby galneon » Wed Jul 16, 2014 5:11 pm

Hahah, that's great. Thank you. Very little time wasted and I'm glad to contribute in some tiny way to this amazing program.

sheppaul
Posts: 24
Joined: Sat Jan 12, 2013 5:58 pm

Re: Keep offline volumes in the index

Postby sheppaul » Thu Jul 17, 2014 1:33 am

It works great.

By the way, I found a weird operation.

I have four hdds keeping offline volumes and these index has gone when I connect usb-hdd.
Those index data of W, X, Y, Z drives seems to be lost while rebuilding the index of a newly attached usb-hdd.

always online: C, D, E, T

keep offline: W, X, Y, Z (built-in hdd rack)

usb-hdd: K

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Thu Jul 17, 2014 2:31 am

Added:
  • Offline indexes will be lost when the index is rebuilt. All volumes must be online when Everything builds its index.

Unfortunately, the offline indexes will be lost when the indexes are rebuilt. Once the K: usb-hdd is included it shouldn't happen again. I have plans to fix this in a future release..

sheppaul
Posts: 24
Joined: Sat Jan 12, 2013 5:58 pm

Re: Keep offline volumes in the index

Postby sheppaul » Thu Jul 17, 2014 4:25 am

Thank you for your consideration.

I have one question though.

I'm not sure but searching offline index feels a bit slower comparing performance when the very volumes are online state.

Is there any perceivable limitation of loading speed of list index with the number of index? (I have index over 1 million)

How to turn off the option, "sort by"? I'd like to compare the loading speed of the option but it seems not to disable the option.
Last edited by sheppaul on Thu Jul 17, 2014 4:28 am, edited 1 time in total.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Thu Jul 17, 2014 4:27 am

Everything will attempt to load file dates, sizes and attributes for the displayed files. This might cause a delay when displaying the results that are offline.

sheppaul
Posts: 24
Joined: Sat Jan 12, 2013 5:58 pm

Re: Keep offline volumes in the index

Postby sheppaul » Thu Jul 17, 2014 4:32 am

Thanks for quick answer.

It makes sense.

But what if the columns of file dates, sizes and attributes are turned off?

Actually I don't use it for my purpose. Is there any room to speed up?

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Thu Jul 17, 2014 4:36 am

How to turn off the option, "sort by"? I'd like to compare the loading speed of the option but it seems not to disable the option.


There is no option to disable the sort by option.
You can change the default sort in Tools -> Options -> Home.

But what if the columns of file dates, sizes and attributes are turned off?


If you disable these columns Everything will not attempt to load this information.
However, Everything will still attempt to load the icons if they are visible, which will also cause the delay issue.

Actually I don't use it for my purpose. Is there any room to speed up?

Not at this stage, sorry.

sheppaul
Posts: 24
Joined: Sat Jan 12, 2013 5:58 pm

Re: Keep offline volumes in the index

Postby sheppaul » Thu Jul 17, 2014 4:40 am

It's enough.

everything is really great and I've never seen such a thing since I started my computer life.

Thank you again. :mrgreen:

Biff
Posts: 34
Joined: Mon May 25, 2015 7:09 am

Re: Keep offline volumes in the index

Postby Biff » Thu May 28, 2015 6:32 am

I am not quite sure to understand this thread. The settings I have tried do not work for me.

I would like to always keep all of my (external) drives (plugged off or not) in the index. And I would like to have not any double entries in the search results. How would I have to do the settings in Everything?

salazor
Posts: 226
Joined: Tue Jun 17, 2014 10:52 am

Re: Keep offline volumes in the index

Postby salazor » Fri May 29, 2015 1:17 pm

Everything indexes only file names and attributes option keep_missing_indexes=1 works well, but I propose you to do so: Add any external disks as .efu ie Menu | Tools | File lists editor farther you will know - then Options | File list and add files .efu - you will have a file name, size, attributes, modification and creation date.

Remember that there was no duplicates keep_missing_indexes=0

Biff
Posts: 34
Joined: Mon May 25, 2015 7:09 am

Re: Keep offline volumes in the index

Postby Biff » Fri May 29, 2015 5:53 pm

Thank you, salazor.

and attributes option keep_missing_indexes=1 works well

I do not understand it, each time you plug in / off a drive Everything creates a new index and after the files of the external drives plugged off are not listed / shown anymore. How could one avoid it?

Add any external disks as .efu ie Menu | Tools | File lists editor farther you will know - then Options | File list and add files .efu - you will have a file name, size, attributes, modification and creation date.

I might have understand it right. And those lists automatically are indexed / re-created / updated when a drive is plugged in? And what is the difference to "Indexes", tab "Folders"?

And how could I avoid Everything to show the same file more than once? When an external drive is plugged in and automatically indexed Everything shows all of the files / folders twice.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Sun May 31, 2015 7:12 am

I do not understand it, each time you plug in / off a drive Everything creates a new index and after the files of the external drives plugged off are not listed / shown anymore. How could one avoid it?


This will be fixed in the next release of Everything.
For now please consider file lists.

I might have understand it right. And those lists automatically are indexed / re-created / updated when a drive is plugged in?


File lists were initially designed for static file lists, for example, a CD-Rom.
File lists must be manually recreated and the index must be manually rebuilt.
I hope to add file list change monitoring in the next release.

And what is the difference to "Indexes", tab "Folders"?


The same issue you have with NTFS volumes will occur with folder indexing.
When the index is rebuilt and the volumes are offline, the folder indexes will appear empty until the volume is online.
This will be fixed in the next update.

And how could I avoid Everything to show the same file more than once? When an external drive is plugged in and automatically indexed Everything shows all of the files / folders twice.


Remove duplicated indexes you may have in the NTFS tab or Folders tab.
NTFS volumes are automatically included, so please try removing the folders index first.

Biff
Posts: 34
Joined: Mon May 25, 2015 7:09 am

Re: Keep offline volumes in the index

Postby Biff » Sun May 31, 2015 10:23 pm

Many thanks, void,

So I can't wait for the next update, that sounds very great.

Code: Select all

Remove duplicated indexes you may have in the NTFS tab or Folders tab.
NTFS volumes are automatically included, so please try removing the folders index first.

Actually I meant without removing such an index. So after (or before) a drive is plugged in I would have to remove the folder / NTFS indexes and before plugging it off I would have to add those indexes since I want them during the time the drive is pugged off. That might be a bit circumstantial at the moment, but after that fix in the next release it won't be necessary anyway, if I understand it right.

Alright, I assume I have not yet entirely understood the difference between using the file lists, folder and NTFS indexes, but for now I know how I can use them to handle the unplugged drives.

Many thanks again.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Mon Jun 01, 2015 6:53 am

Actually I meant without removing such an index. So after (or before) a drive is plugged in I would have to remove the folder / NTFS indexes and before plugging it off I would have to add those indexes since I want them during the time the drive is pugged off. That might be a bit circumstantial at the moment, but after that fix in the next release it won't be necessary anyway, if I understand it right.


There is currently no way around the duplication when you have a NTFS and folder index of the same volume.
The next update should remove the need for adding duplicated volumes in the first place.

Biff
Posts: 34
Joined: Mon May 25, 2015 7:09 am

Re: Keep offline volumes in the index

Postby Biff » Mon Jun 01, 2015 8:59 am

Alright, I understand, cannot wait for the update, many thanks, void.

myNameIsJonny!
Posts: 12
Joined: Mon Mar 16, 2015 2:12 pm

Re: Keep offline volumes in the index

Postby myNameIsJonny! » Sun Nov 29, 2015 10:46 pm

Hi, just want to clarify something, I want to keep external hard-drives indexed in my EXT db, using keep_missing_indexes

I'm running everything portable amongst my multiple builds, so C has different GUID.

Auto include new removable volumes
Automatically include new removable volumes in Everything.
Everything must rebuild the database when a volume path, volume guid, the USN Journal ID changes or the volume goes offline.
See the Everything.ini option keep_missing_indexes to keep offline volumes in the index.



I see some volume info in .ini, even after unticking options for C: I have the following

ntfs_volume_paths="C:","D:","F:","L:","G:","H:","I:","T:","K:"
ntfs_volume_includes=0,0,1,1,1,1,0,1,1
ntfs_volume_load_recent_changes=0,0,0,0,0,0,0,0,0
ntfs_volume_include_onlys="","","","","","","","",""
ntfs_volume_monitors=0,1,1,1,1,1,1,1,1


Should I remove the C volume path ^^?
Do I need to edit any of the other related settings (volume_monitors etc- counting now I see there is a match, so I guess if I remove the first parameter from each, that'll reflect the C volume NOT being indexed?

Will I then be able to use a communal database pointing to indexed (but disconnected) volumes? (providing they are assigned another letter, or another volume is assigned their letter etc)?

Thanks!

skan
Posts: 6
Joined: Wed Aug 12, 2015 8:40 pm

Re: Keep offline volumes in the index

Postby skan » Fri Jan 15, 2016 10:43 am

Hello

Is it also working for usb pendrives?

gdmellott
Posts: 1
Joined: Mon Mar 07, 2016 11:38 am

Re: Keep offline volumes in the index

Postby gdmellott » Mon Mar 07, 2016 12:23 pm

I wonder why the "rebuilding" of the index needs to require the program to stop. Why not have the index search and viewer use the old index until the new one is made and put a blinking status bar message, or popup warning, stating that the index still is being refined.

The ideal goal for this program would be for it to maintain an complete an index of all of ones file, whether they were online or not. Noting their online/offline status with a differing color or shade of text. Perhaps there should be a simple button to skip displaying offline files, should that generate too many, or otherwise not be desired at the time. What presently makes things hard, is dealing with a 4TB backup drive one would prefer to not have on all time, or otherwise need to leave behind for a while, or perhaps suspend the computer, etc. and thus find it needs to remake the index all over again. One waits a long time for it to get done so one can use the program's index searching functionality again.

Frankly, I'd be interested in noting if I have files that have gone missing, particularly out of certain directories and there subs.

Also, I'm not sure how you go about updating the index, yet it is my understanding that one only needs to note the date each folder was last modified to determine if it needs more checking of each file to find the ones modified. The updating works fast, so that may be the method used. In any case, I have near nothing against having a sub index for a given drive stored on the drive itself, should that make things simpler for working with removable drives that wonder between more than one computer.

myNameIsJonny!
Posts: 12
Joined: Mon Mar 16, 2015 2:12 pm

Re: Keep offline volumes in the index

Postby myNameIsJonny! » Wed Apr 20, 2016 1:09 pm

gdmellott wrote:I wonder why the "rebuilding" of the index needs to require the program to stop. Why not have the index search and viewer use the old index until the new one is made and put a blinking status bar message, or popup warning, stating that the index still is being refined.

The ideal goal for this program would be for it to maintain an complete an index of all of ones file, whether they were online or not. Noting their online/offline status with a differing color or shade of text. Perhaps there should be a simple button to skip displaying offline files, should that generate too many, or otherwise not be desired at the time. What presently makes things hard, is dealing with a 4TB backup drive one would prefer to not have on all time, or otherwise need to leave behind for a while, or perhaps suspend the computer, etc. and thus find it needs to remake the index all over again. One waits a long time for it to get done so one can use the program's index searching functionality again.

Frankly, I'd be interested in noting if I have files that have gone missing, particularly out of certain directories and there subs.

Also, I'm not sure how you go about updating the index, yet it is my understanding that one only needs to note the date each folder was last modified to determine if it needs more checking of each file to find the ones modified. The updating works fast, so that may be the method used. In any case, I have near nothing against having a sub index for a given drive stored on the drive itself, should that make things simpler for working with removable drives that wonder between more than one computer.


if that's a FR, ya got my vote :)

Zavod
Posts: 1
Joined: Tue May 31, 2016 3:15 pm

Re: Keep offline volumes in the index

Postby Zavod » Tue May 31, 2016 3:26 pm

This topic is the reason I signed up for an account...

So I have this requirement:

I have a drive that frequently goes dormant when idle for more than 30 minutes (a backup drive) and occasionally Everything will have to re-index. Inconveniently, it's a huge USB 2.0 drive array, a Drobo, so it takes on the order of 10+ minutes to complete the indexing. During that time, Everything is not usable.

I enabled "keep_missing_indexes=1" just now, and that does solve appear to solve the disconnect problem. (yay!)

BUT...

I have *another* drive, with transient data on it. When it mounts, I want to index it. But when it unmounts, I want it to go away. I tested just now, and it's data appear to stay around now that "keep_missing_indexes=1".

So to summarize:
I would like *some* drives to stay indexed with disconnected, and some *other* drives's data discarded when disconnected.

Is there a way to accomplish that?
(loving the program - only had it a month, and can't live without it... sent a donation today!)

Chender
Posts: 5
Joined: Wed Dec 29, 2010 10:03 pm

Re: Keep offline volumes in the index

Postby Chender » Wed Aug 03, 2016 9:03 pm

Hi!
Sometimes I don't want to use the datas of my removable drives. Then they aren't in my computer. How can I generate a new index file without them?

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Sat Aug 06, 2016 5:55 am

Sometimes I don't want to use the datas of my removable drives. Then they aren't in my computer. How can I generate a new index file without them?


In Everything 1.4, to automatically remove offline volumes:
  • In Everything, from the Tools menu, click Options.
  • Click the NTFS tab on the left
  • Check Automatically remove offline volumes.
  • Click OK.

To manually remove offline volumes:
  • In Everything, from the Tools menu, click Options.
  • Click the NTFS tab on the left
  • Select the offline volume.
  • Click Remove.
  • Click OK.

Chender
Posts: 5
Joined: Wed Dec 29, 2010 10:03 pm

Re: Keep offline volumes in the index

Postby Chender » Sat Aug 06, 2016 9:12 pm

Thanks a lot!

Marc2222
Posts: 10
Joined: Sat Nov 14, 2015 8:01 pm

Re: Keep offline volumes in the index

Postby Marc2222 » Sun Dec 04, 2016 8:08 pm

It is not working here... I set the line in the ini file but it doesn't display results for offline volumes. What is odd is that I disconnected a drive with the letter "I:" and connected another drive with the same letter. After that results for the first "I" drive are displayed even though is not the same drive... I'm using version 1.3.4.686 (x86) with the read only parameter.

void
Site Admin
Posts: 3046
Joined: Fri Oct 16, 2009 11:31 pm

Re: Keep offline volumes in the index

Postby void » Mon Dec 05, 2016 12:00 am

When Everything is in read only mode, offline files are always displayed.
If the offline files are not displayed, then they have been lost.
You will need to bring these offline volumes online and re-index.


Return to “Support”