database always corrupt

If you are experiencing problems with "Everything", post here for assistance.
Post Reply
davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

database always corrupt

Post by davidicus » Sat Apr 11, 2020 8:07 pm

every time i restart or awaken my new laptop (maybe once a day?), Everything reports the database is corrupt and needs to rescan. i've tried numerous rescans, uninstalling and reinstalling with and without default options. it's always worked great on the old, very similar, laptop with the same network drives, and setup. any ideas appreciated.

NotNull
Posts: 2283
Joined: Wed May 24, 2017 9:22 pm

Re: database always corrupt

Post by NotNull » Sun Apr 12, 2020 2:04 pm

Everything doesn't use the database on disk when it is running. All "magic" happens in RAM. So this is a bit strange.
What is the exact thing that Everything is reporting about corruption? Can you post a screenshot?


Furthermore: Windows version? Everything version? 32-bit or 64-bit.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Sun Apr 12, 2020 11:05 pm

thanks! today it happened again so I grabbed a screenshot. Everything was running in the background.
  • Everything v1.4.1.969 64
  • Windows 10 64 Home v1903, Intel i7-8565U CPU @ 1.8 GHz, 16 GB RAM
EverythingError 2020-04-12 113037.png
EverythingError 2020-04-12 113037.png (9.76 KiB) Viewed 708 times

NotNull
Posts: 2283
Joined: Wed May 24, 2017 9:22 pm

Re: database always corrupt

Post by NotNull » Mon Apr 13, 2020 3:35 pm

Thanks for the info!

Let's start by deleting Everything's database, so that it can start fresh:
  • Make sure Everything is not running:
    Right-click the Everything icon in the system-tray and choose exit
  • In File Explorer, enter
    %localappdata%\Everything
    in the address bar and press ENTER
  • Delete Everything.db
  • Done
After these steps you can test if that solved your issue.
If not: there are more options, but let's take it step by step.

horst.epp
Posts: 279
Joined: Fri Apr 04, 2014 3:24 pm

Re: database always corrupt

Post by horst.epp » Mon Apr 13, 2020 5:05 pm

The Everything.db may also be in the Everything program dir
if a portable installation is used.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Mon Apr 13, 2020 6:37 pm

Ok, I don't recall using a portable installation, but there's nothing in AppData\Local\Everything. Directory is there, but empty.
Program Files/Everything shows a number of files, including Everything.exe, Everything.ini, and Everything.lng, but no Everything.db.
Weird?

NotNull
Posts: 2283
Joined: Wed May 24, 2017 9:22 pm

Re: database always corrupt

Post by NotNull » Mon Apr 13, 2020 7:20 pm

davidicus wrote:
Mon Apr 13, 2020 6:37 pm
Weird?
Not really. Everything keeps it's database in RAM until you exit the program. Apparenttly the database has never been written to disk successfully.

Before asking you to scan your local drives for error (this is a new laptop, so diskerrors are not high on the list) first:
  • Start Everything
  • Go to Menu:Tools > Options > General
  • Make sure Store settings and data in %APPDATA%\Everything is enabled
  • Make sure Everything Service is enabled
  • Make sure Run as administrator is disabled
  • Click the OK button
  • Wait until Everything has finished scanning the disks (see status bar)
  • Exit Everything (Menu:File > Exit)
  • Check if Everything.db is written to %localappdata%\Everything (what is the size?)
  • Start Everything
  • See if the issue comes back

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Mon Apr 13, 2020 7:58 pm

Ok, scanning now. Thank you.
I didn't have to change any settings; they matched what you specified.
Took a while to scan. Everything.db is in AppData/Local/Everything at about 41 MB.

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Tue Apr 14, 2020 4:04 am

Everything will report the database as corrupt if it is 'no longer sorted'.

This can usually occur if the Everything.db was corrupted on disk or in memory.

Please make sure Everything completely exits when you shutdown your computer. (Do not force shutdown)

Please make sure Everything is storing data and settings on your C: drive:
  • In Everything, from the Tools menu, click Options.
  • Click the General tab on the left.
  • Check Store settings and data in %APPDATA%\Everything
  • Click OK.
Please check your disks for errors:
  • Please backup anything important on your drive before checking it for errors.
  • In Windows Explorer, right click your drive and click Properties.
  • Click the Tools tab.
  • Click Check now...
  • Click Start
  • If prompted to schedule a scan on the next restart, click Yes.
Please check your memory.

There could also be something wrong with Everything handling file system changes.
What volumes do you have indexed with Everything? Are they all NTFS volumes (under Tools -> Options -> NTFS)? -Or have you added any folder indexes under Tools -> Options -> Folders?

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Tue Apr 14, 2020 8:14 pm

The error just appeared again. Same error, 12023. Everything was open at the time.
- store settings and data: check
- memory: check (took a long while)
- hard drive: I didn't scan, because you said when Everything is open the db is in RAM
- I've added two NAS volumes

at first I thought I had set up the NAS folders the same as on the old machine, but when I look now I can see two differences
- old machine folders are listed as \\LG-NAS\service\backup instead of S:\backup, so maybe I added differently
- old machine is not set to attempt to monitor changes, but new machine is. I should probably deselect that?

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Thu Apr 16, 2020 3:32 am

Could you please send me your debug logs:
  • In Everything, type in the following search and press ENTER:
    /debug_log
  • Wait for the database is corrupt message.
  • Please zip and send your %TEMP%\Everything Debug Log.txt file to support@voidtools.com
After sending the debug log, please try disabling monitor changes and see if it helps.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Sun Apr 19, 2020 11:05 pm

So, I had already disabled Monitor Changes and stopped getting the error, so I kept on truckin' until today, when I just got the error again (a good long run, comparatively). I'm fairly certain I had searched for /debug_log as you said, if that's a critical step. There is no log file, though (AppData/Local/Everything is empty).

I guess I'll try going through the process again.

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Mon Apr 20, 2020 3:48 am

Thank you for your reply.

Debug logging will need to be re-enabled every time you restart your system or restart Everything.

There is a way to turn on debug logging that will persist over system restarts.
However, I try to avoid this as it can end up writing a lot of text to the log (1GB+ logs).

The log file should be stored in %TEMP%
This should expand to:
C:\Users\<your user name>\AppData\Local\Temp
The log file is always called Everything Debug Log.txt
The log is always cleared when you restart your system or restart Everything with debug logging enabled.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Mon Apr 20, 2020 6:22 pm

great--that clarifies quite a bit, thanks. on it.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Wed Apr 22, 2020 6:09 pm

got a log! sent as requested

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Fri Apr 24, 2020 12:00 am

Thank you for the debug logs.

I suspect Everything is finding a 'folder index' file multiple times. (FindFirstFile is returning the same file multiple times)
This would cause database corruption.

Could you please try the latest nightly build of Everything.
This version will ignore duplicated FindFirstFile files.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Fri Apr 24, 2020 12:19 am

On it. Thank you!

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Sun Apr 26, 2020 3:48 am

bad news. same error today :/

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Tue Apr 28, 2020 1:08 am

There may have been a duplicate in the index from the previous version.

Please let me know if you see this error again.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Wed Apr 29, 2020 10:30 pm

Database is corrupt again. I have another debug log if you think it will help?

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Wed Apr 29, 2020 10:33 pm

Yes, please send to support@voidtools.com

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Tue May 05, 2020 2:50 am

Hi--new corruption, new log. Sent by email. <fingers crossed>

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Sun May 10, 2020 5:23 am

Thanks for the debug logs.

The debug logs show Everything found duplicated filenames on a NAS share.
Everything is mixing up references to duplicated filenames, causing database corruption.

For anyone having trouble with corrupt db message, please try the latest nightly build.

This version will fix the corrupt db error caused by duplicated filenames.
Duplicated files are removed during index creation.

Please make sure you delete your Everything.db before running this version:
  • Completely exit Everything (right click the Everything system tray icon and click Exit).
  • Delete your %LOCALAPPDATA%\Everything\Everything.db
  • From the Start menu, type in:
    services.msc
  • Right click Services and click Run as administrator.
  • Right click Everything and click Stop Service.
  • Overwrite your existing Everything.exe with one of the zipped Everything.exe below.
  • In Services, Right click Everything and click Start Service.
  • Run Everything.exe
Please let me know if you see a corrupt db error message with this version.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Mon May 11, 2020 8:11 pm

Thanks very much! So far, so good.
I deleted that whole ancient Mac trash directory, too. It never should have been archived in the first place.
I hope it helped make Everything stronger, in any case.

BTW, any idea why the new machine with Everything choked on it, but the old one didn't? The same network drive was being indexed.

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Wed May 13, 2020 1:10 am

Hard to say.

It all depends on Everything's file reference.
There's a 50/50 chance Everything is referring to the correct file when the file is duplicated.

It might have something to do with long file name lookup on the new system.
From the logs it was always a short name that was duplicated, eg:
______~X
______~X



How the corruption occurs:
Everything tries to delete the first ______~X file above. However, the second one ends up being removed from the index.
Everything tries to delete the next ______~X file above. However, it was already removed and the corrupt db message is shown.

davidicus
Posts: 14
Joined: Sat Apr 11, 2020 7:59 pm

Re: database always corrupt

Post by davidicus » Sat May 23, 2020 11:50 pm

Been working great for a while now. Thank you!
BTW, the application that seemed to be bulking up the log looked like Dropbox. Appreciate that heads up.

void
Site Admin
Posts: 5652
Joined: Fri Oct 16, 2009 11:31 pm

Re: database always corrupt

Post by void » Mon May 25, 2020 1:38 am

Glad to hear Everything is working now!

Debug logs can grow very large.

To disable the debug logging in Everything:
  • In Everything, type in the following search and press ENTER:
    /debug_log=0
  • Restart Everything:
  • Type in the following search and press ENTER:
    /restart
The Everything restart will also stop debug logging that was started with the search command: /debug_log

Post Reply