Memory leak with property indexing

Discussion related to "Everything" 1.5 Alpha.
Post Reply
JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Memory leak with property indexing

Post by JTCGiants56 » Mon Nov 22, 2021 2:44 am

It seems I'm still experiencing this memory bug after doing a full rebuild and index on the newest build. The memory usage will jump up to 8gb usage when indexing properties, but even after it is finished is not flushed down to normal memory usage levels. Until I restart everything then it goes back to normal levels.

I can start a new thread for this if easier.
Attachments
Screenshot 2021-11-21 214305.png
Screenshot 2021-11-21 214305.png (23.2 KiB) Viewed 7389 times

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Mon Nov 22, 2021 5:11 am

Thank you for the bug report JTCGiants56,

I have moved your thread to a new topic.



Could you please let me know what properties you are indexing.

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Mon Nov 22, 2021 5:17 am

void wrote:
Mon Nov 22, 2021 5:11 am
Thank you for the bug report JTCGiants56,

I have moved your thread to a new topic.



Could you please let me know what properties you are indexing.
I am indexing just these filetypes:

Code: Select all

*.mkv;*.mp4;*.avi;*.flv;*.webm;*.wmv;*.mov;*.m4v;*.m2ts;*.ts;*.mpg
For these properties:
Screenshot 2021-11-22 001634.png
Screenshot 2021-11-22 001634.png (6.47 KiB) Viewed 7344 times

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Wed Nov 24, 2021 8:35 am

Thanks for the information JTCGiants56,

I have fixed some minor leaks for the next update.
I haven't found any major leaks yet.

I suspect there might be a leak in a shell extension that is used to gather properties for wmv, mov, m4v, m2ts, ts or mpg.
I haven't found a major leak with the default Windows 10 shell extensions for these file types.

Are you using any third party shell extensions to read properties?

I will add support for mov and m4v in the next alpha update.


Support for wmv, m2ts, ts or mpg is on my TODO list.



Please try temporarily removing wmv, mov, m4v, m2ts, ts and mpg from your indexed properties, force a rebuild and see if the issue persists.

I'm working on adding more memory debug information.

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Wed Nov 24, 2021 9:19 pm

void wrote:
Wed Nov 24, 2021 8:35 am
Thanks for the information JTCGiants56,

I have fixed some minor leaks for the next update.
I haven't found any major leaks yet.

I suspect there might be a leak in a shell extension that is used to gather properties for wmv, mov, m4v, m2ts, ts or mpg.
I haven't found a major leak with the default Windows 10 shell extensions for these file types.

Are you using any third party shell extensions to read properties?

I will add support for mov and m4v in the next alpha update.


Support for wmv, m2ts, ts or mpg is on my TODO list.



Please try temporarily removing wmv, mov, m4v, m2ts, ts and mpg from your indexed properties, force a rebuild and see if the issue persists.

I'm working on adding more memory debug information.
I used ShellExView then sorted by type. The only third party property handler I found was something for macrium reflect.

Code: Select all

ReflectPropertyHandler.PropertyHandler	No		Property Handler	Macrium Reflect Shell Extensions	8, 0, 6392, 0	Macrium Reflect Shell Extensions	Paramount Software UK Ltd	No	No	No	No	No	C:\Program Files\Macrium\Reflect\RShellEx.dll	No	{41FC0A91-1EDB-40D6-8858-AA642AE45737}	11/17/2021 8:59:22 PM	11/19/2021 2:08:02 AM	No	A	10,622,192	No		No	
After removing those properties big improvement. This is the memory usage after doing a full rebuild.
Screenshot 2021-11-24 161518.png
Screenshot 2021-11-24 161518.png (11.69 KiB) Viewed 7125 times
After restarting everything after that usage goes even lower, not sure if that is a tiny leftover leak, but much much better than the 8gb memory I was using before.
Screenshot 2021-11-24 161533.png
Screenshot 2021-11-24 161533.png (11.71 KiB) Viewed 7125 times

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Tue Nov 30, 2021 4:38 am

Thank you for testing JTCGiants56,

Everything 1.5.0.1286a fixes a minor memory leak when reindexing.
Everything 1.5.0.1286a adds support for mov and m4v.

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Tue Nov 30, 2021 6:32 am

void wrote:
Tue Nov 30, 2021 4:38 am
Thank you for testing JTCGiants56,

Everything 1.5.0.1286a fixes a minor memory leak when reindexing.
Everything 1.5.0.1286a adds support for mov and m4v.
Awesome, thanks!

I noticed this build [Administrator] started appearing on the window on top in this build and the administrator setting was set to "intermediate" mode.

I guess I'm confused about the new intermediate mode vs unchecked. Both say "run as a standard user". I believe before I was just running the service only, which is how I have it in 1.4
21221.png
21221.png (25.18 KiB) Viewed 6113 times

raccoon
Posts: 722
Joined: Thu Oct 18, 2018 1:24 am

Re: Memory leak with property indexing

Post by raccoon » Tue Nov 30, 2021 6:49 am

The Indeterminate checkbox state means that some other process, user action or external influence is causing Everything to run with elevated privileges (As Administrator). Even though the configuration is set to Standard User.

If you are launching Everything from a Windows Shortcut, or Task Scheduler, or AutoHotkey Script, or Windows Explorer window that is run As Administrator, then Everything will inherit the same As Administrator elevated privilege.

Hope that helps and gives you an idea of what might be causing it.

Glad my suggestion to add [Administrator] to the titlebar is already helping people! (strokes ego) It was just added to tonight's release as a new feature. That means you've probably been running Everything as Administrator for a while now without realizing it.
Last edited by raccoon on Tue Nov 30, 2021 6:58 am, edited 1 time in total.

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Tue Nov 30, 2021 6:57 am

raccoon wrote:
Tue Nov 30, 2021 6:49 am
Intermediate means that some other process, user action or external influence is causing Everything to run with elevated privileges (As Administrator). Even though the configuration is set to Standard User.

If you are launching Everything from a Windows Shortcut, or Task Scheduler, or AutoHotkey Script, or Windows Explorer window that is run As Administrator, then Everything will inherit the same As Administrator elevated privilege.

Hope that helps and gives you an idea of what might be causing it.

Glad my suggestion to add [Administrator] to the titlebar is already helping people! (strokes ego) It was just added to tonight's release as a new feature. That means you've probably been running Everything as Administrator for a while now without knowing it.
Thanks, that helps. I am simply searching for everything in the start menu. It looks like what comes up is a shortcut. I guess it runs shortcuts as admin by default.


@void
It looks like one of the properties other than m4v and mov is still causing the memory leak for me. I will remove those for the time being. Thanks
yyyy.png
yyyy.png (12.76 KiB) Viewed 6099 times

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Wed Dec 01, 2021 8:58 am

Please try right clicking your Everything.exe and disabling Run this program as an administrator:
  • Right click your Everything.exe or a shortcut to Everything.
  • Click Properties.
  • Click the Compatibility tab.
  • Uncheck Run this program as an administrator.
  • Click OK.
It looks like one of the properties other than m4v and mov is still causing the memory leak for me. I will remove those for the time being. Thanks
Thanks for the info.
I wonder what property / file is causing this.
I wonder if its worth adding one property at a time to try and find the offending one?

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Thu Dec 02, 2021 1:44 am

void wrote:
Wed Dec 01, 2021 8:58 am
Please try right clicking your Everything.exe and disabling Run this program as an administrator:
  • Right click your Everything.exe or a shortcut to Everything.
  • Click Properties.
  • Click the Compatibility tab.
  • Uncheck Run this program as an administrator.
  • Click OK.
This was actually the first thing I checked. In the shortcut used by the startmenu and directly on the everything.exe file but it was unchecked. Also wasn't checked under the "change setting for all users option". Not a big deal either way for me.
It looks like one of the properties other than m4v and mov is still causing the memory leak for me. I will remove those for the time being. Thanks
Thanks for the info.
I wonder what property / file is causing this.
I wonder if its worth adding one property at a time to try and find the offending one?
I can give this a go this weekend when I have some free time. Unfortunantly I have a lot of files so each rescan takes a decent amount of time.

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Thu Dec 02, 2021 2:15 am

Also to run Everything as a standard user:
  • Right click your Everything.exe or Everything shortcut.
  • Click Properties.
  • Click the Shortcut tab.
  • Click the Advanced button.
  • Uncheck Run as administrator.
  • Click OK.
  • Click OK.
Check if there is a scheduled task starting Everything on startup as an admin.
(or other program starting Everything on startup as admin)


You should be able to produce the same leak simply by searching for (or sorting by) the property.

For example:

ext:mpg frame-rate:
ext:mpg height:
ext:mpg length:
ext:mpg media-created:
ext:mpg width:

See if one property is causing the leak, if so, try reducing the search pool to a single drive, then folder, eg:

Restart Everything and search for:
c: ext:mpg frame-rate:

Restart Everything and search for:
c:\media\ ext:mpg frame-rate:

Restart Everything and search for:
c:\media\folder1 ext:mpg frame-rate:

It might be possible to find the exact file causing the leak.

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Sat Dec 25, 2021 5:56 am

void wrote:
Thu Dec 02, 2021 2:15 am
Also to run Everything as a standard user:
  • Right click your Everything.exe or Everything shortcut.
  • Click Properties.
  • Click the Shortcut tab.
  • Click the Advanced button.
  • Uncheck Run as administrator.
  • Click OK.
  • Click OK.
Check if there is a scheduled task starting Everything on startup as an admin.
(or other program starting Everything on startup as admin)
I just checked and it is not set to run as admin in the exe (even in the set for all users section). Even if I fully uncheck the run as admin box, then click ok to restart as standard user, it always opens up rechecked like below.
Screenshot 2021-12-25 005304.png
Screenshot 2021-12-25 005304.png (3.75 KiB) Viewed 3082 times
void wrote:
Thu Dec 02, 2021 2:15 am
You should be able to produce the same leak simply by searching for (or sorting by) the property.

For example:

ext:mpg frame-rate:
ext:mpg height:
ext:mpg length:
ext:mpg media-created:
ext:mpg width:

See if one property is causing the leak, if so, try reducing the search pool to a single drive, then folder, eg:

Restart Everything and search for:
c: ext:mpg frame-rate:

Restart Everything and search for:
c:\media\ ext:mpg frame-rate:

Restart Everything and search for:
c:\media\folder1 ext:mpg frame-rate:

It might be possible to find the exact file causing the leak.
Sorry for delay on troubleshooting this, I will dedicate some time following the holidays.

I will try your quicker method, but I notice a total property reindex is initiated if a single extension is removed from the line. For example, if all I do is remove the wmv extension from my include list, it reindexes all the properties for the other extensions again, even though nothing changed for them.

raccoon
Posts: 722
Joined: Thu Oct 18, 2018 1:24 am

Re: Memory leak with property indexing

Post by raccoon » Sat Dec 25, 2021 6:41 am

If the Run As Admin box is in a mid state, that means that you are running Everything as Administrator yourself, Everything is not doing it.

Check your start menu shortcuts and compatibility settings to make sure they are not set to Run As Administrator. If you are launching Everything from Windows Task Scheduler, check in there too. If you are launching Everything from an AutoHotkey script, or any other launcher, then it will inherit the privilege of that process.

explorer_rfv3pWqSqX.png
explorer_rfv3pWqSqX.png (41.39 KiB) Viewed 3075 times

JTCGiants56
Posts: 158
Joined: Fri Nov 28, 2014 3:58 pm

Re: Memory leak with property indexing

Post by JTCGiants56 » Sat Dec 25, 2021 5:37 pm

raccoon wrote:
Sat Dec 25, 2021 6:41 am
If the Run As Admin box is in a mid state, that means that you are running Everything as Administrator yourself, Everything is not doing it.

Check your start menu shortcuts and compatibility settings to make sure they are not set to Run As Administrator. If you are launching Everything from Windows Task Scheduler, check in there too. If you are launching Everything from an AutoHotkey script, or any other launcher, then it will inherit the privilege of that process.


explorer_rfv3pWqSqX.png
Hmm, I'm stumped. I always open 1.5 by searching it from the start menu. Checked C:\ProgramData\Microsoft\Windows\Start Menu\Programs where the shortcut resides and its not set as admin. Searched everything for everything shortcuts and checked every one, not have it set.

I am on windows 11 now so will do some more research, there may be something else going on with auto running as admin somehow.

void
Developer
Posts: 9627
Joined: Fri Oct 16, 2009 11:31 pm

Re: Memory leak with property indexing

Post by void » Wed Dec 29, 2021 5:36 am

Everything might be starting as an administrator on startup.

To confirm:
  • In Everything, from the File menu, click Exit.
  • Launch Everything from the Start menu.
  • Does Everything run as an administrator?

Post Reply