Is there is away to generate folder hash ?
-
- Posts: 25
- Joined: Mon Aug 07, 2023 10:57 pm
Is there is away to generate folder hash ?
Hello , I see this option only on Treesize , using it for long time but it's very slow but lacks so many feature and it's long time to get a feature to be added
So is it possible to add a folder hash calculation so we can see it on the column as it only works for the files at the moment.
my plan is to get a folder dupe option while exclude such a file type , this will save me tons of time , I have been working on project sorting tens of terabytes of date I get lost after ran out of options now I have 3.6 TB that needed to be sorted and I'm doing it manually because there is folders have same names but files are inside and different .
So if i get to exclude one file and using some kind of md5 files option next to dupe: this will save me like 2 years at least of work . noway to risk deleting the wrong folder or not 100 % exact folder , also if there is a better way please let me know.
Thank you.
So is it possible to add a folder hash calculation so we can see it on the column as it only works for the files at the moment.
my plan is to get a folder dupe option while exclude such a file type , this will save me tons of time , I have been working on project sorting tens of terabytes of date I get lost after ran out of options now I have 3.6 TB that needed to be sorted and I'm doing it manually because there is folders have same names but files are inside and different .
So if i get to exclude one file and using some kind of md5 files option next to dupe: this will save me like 2 years at least of work . noway to risk deleting the wrong folder or not 100 % exact folder , also if there is a better way please let me know.
Thank you.
Re: Is there is away to generate folder hash ?
Do you know if the free version of Treesize is able to hash folders?
Thanks.
Thanks.
-
- Posts: 25
- Joined: Mon Aug 07, 2023 10:57 pm
Re: Is there is away to generate folder hash ?
7z has an option to calculate the folder hash.
A feature to do this in Everything is on my TODO list.
A feature to do this in Everything is on my TODO list.
Re: Is there is away to generate folder hash ?
Thanks all...
-
- Posts: 25
- Joined: Mon Aug 07, 2023 10:57 pm
Re: Is there is away to generate folder hash ?
identifying duplicates from checksums7z has an option to calculate the folder hash.
Be careful with that - as the way a directory is parsed will make a difference.
If you always use the same tool, & that tool does not change, it shouldn't matter. Otherwise...
-
- Posts: 25
- Joined: Mon Aug 07, 2023 10:57 pm
Re: Is there is away to generate folder hash ?
therube wrote: ↑Mon Oct 09, 2023 2:43 pmidentifying duplicates from checksums7z has an option to calculate the folder hash.
I have huge problem it's not just a single folder i need to generate a hash for im talking about over 350k just in one drive, over 12 tb of random date a few of them are dupes however some duped folders contains note files but these note files are important and they are the problem that i need to generate hashes for all the folders without scanning the note files , so after i get the result for exact hash folders i will delete one and keep one
some folders . im working on a project for years and still i can't get it done, manually is very slow i can't give up ,
======================================================================================================
Be careful with that - as the way a directory is parsed will make a difference.
If you always use the same tool, & that tool does not change, it shouldn't matter. Otherwise...
everything is my only choice , other tools is helpful to do one job only because it's scanning selected folder , i hope there is a live chat so i can get a quick help
Re: Is there is away to generate folder hash ?
Everything 1.5.0.1358a adds support for folder hashes.
To show the folder hash:
Everything uses the same folder hash algorithm as 7zip 21.04 or later.
folder-data-and-names-crc32:
folder-data-and-names-crc64:
folder-data-and-names-md5:
folder-data-and-names-sha1:
folder-data-and-names-sha256:
folder-data-and-names-sha512:
folder-data-crc32:
folder-data-crc64:
folder-data-md5:
folder-data-sha1:
folder-data-sha256:
folder-data-sha512:
To instantly find folders with a matching size, include the following in your search:
folder:dupe:size
To show the folder hash:
- In Everything 1.5, right click the Result List Column Header and click Add Columns....
- Click the Content tab on the left.
- Select one of the Folder Data * properties and click OK.
Calculating the folder hash will take a very long time. |
Everything uses the same folder hash algorithm as 7zip 21.04 or later.
folder-data-and-names-crc32:
folder-data-and-names-crc64:
folder-data-and-names-md5:
folder-data-and-names-sha1:
folder-data-and-names-sha256:
folder-data-and-names-sha512:
folder-data-crc32:
folder-data-crc64:
folder-data-md5:
folder-data-sha1:
folder-data-sha256:
folder-data-sha512:
To instantly find folders with a matching size, include the following in your search:
folder:dupe:size
Re: Is there is away to generate folder hash ?
Can you link me a spec on 7zip folder hashing, or give me the nutshell version? Is it alphabetical or by size or some combination when multiple files are the same size?
Re: Is there is away to generate folder hash ?
Ah, so a folder-data-sha1: hash only depends on the data that is within a particular tree, & not where within that tree it may be.
(That applies to a top-most directory & to the files/directories within. Obviously if you're looking at subdirectories thereof, & if a particular, identical file is in 1 particular subdirectory in 1 tree, & in a different subdirectory in a different tree, then those 2 subdirectories will not compare [indentical].)
So (assuming the file contents are identical [between two directories]):
c:/1/1.txt c:/1/2.txt c:/1/3.txt
&
c:/2/1.txt c:/2/2.txt c:/2/333/456/3.txt
will give you the same data hash.
That's kind of neat.
If changes are made to a directory, appears you need to Refresh the listing to have Everything pick up on that.
(That applies to a top-most directory & to the files/directories within. Obviously if you're looking at subdirectories thereof, & if a particular, identical file is in 1 particular subdirectory in 1 tree, & in a different subdirectory in a different tree, then those 2 subdirectories will not compare [indentical].)
So (assuming the file contents are identical [between two directories]):
c:/1/1.txt c:/1/2.txt c:/1/3.txt
&
c:/2/1.txt c:/2/2.txt c:/2/333/456/3.txt
will give you the same data hash.
That's kind of neat.
If changes are made to a directory, appears you need to Refresh the listing to have Everything pick up on that.
Re: Is there is away to generate folder hash ?
Hello , I just tested this feature and it does calculate the Hash for excluded file typesvoid wrote: ↑Thu Oct 19, 2023 5:52 am Everything 1.5.0.1358a adds support for folder hashes.
To show the folder hash:
- In Everything 1.5, right click the Result List Column Header and click Add Columns....
- Click the Content tab on the left.
- Select one of the Folder Data * properties and click OK.
Everything uses the same folder hash algorithm as 7zip 21.04 or later.
Calculating the folder hash will take a very long time.
Can you please add an option to void the excluded file types when hashes , I think this is a game changer feature no one ever have this was easy to find lots of files and folders very fast and accurate, Thank you.
- Attachments
-
- 2023-10-19_105953.jpg (33.81 KiB) Viewed 14013 times
Re: Is there is away to generate folder hash ?
Wonder what 7-zip does?
If the above is correct, with the intent to exclude .txt files, 7-zip itself does not seem to take the exclude into consideration.
(Or, I might have an incorrect command-line for what I was wanting to do?)
7zip.exe h ./3/ -scrcSHA1 -x!*.txt
If the above is correct, with the intent to exclude .txt files, 7-zip itself does not seem to take the exclude into consideration.
(Or, I might have an incorrect command-line for what I was wanting to do?)
Re: Is there is away to generate folder hash ?
great job doing this but I just need to exclude few file types while doing the folder dupe hash
need a code or something to write it down the search bar or other idea please
need a code or something to write it down the search bar or other idea please
Re: Is there is away to generate folder hash ?
https://github.com/mcmilk/7-Zip/blob/master/CPP/7zip/UI/Common/HashCalc.cppCan you link me a spec on 7zip folder hashing
These two functions are of interest:
void CHashBundle::Final(bool isDir, bool isAltStream, const UString &path)
static void AddDigests(Byte *dest, const Byte *src, UInt32 size)
In short:or give me the nutshell version?
Calculate the hash for each file and literally add them together.
Name hashes are calculated from the Directory attribute, data hash, path and name.
-Everything and 7zip 21.04 use / as a path separator.
The neat thing is the order of names doesn't matter, you'll still get the same hash sum.
Only if the folder has subfolders and you make a change to a subfolder.If changes are made to a directory, appears you need to Refresh the listing to have Everything pick up on that.
Everything should recalculate the hash if you make a change to a child file/folder.
Press F5 to recalculate the folder hashes.
I will consider calculating hashes from files in the index only.Can you please add an option to void the excluded file types when hashes
I will also consider adding "from-disk" property variants.
Thank you for the suggestions.
Re: Is there is away to generate folder hash ?
Everything 1.5.0.1359a improves folder data and name hashes.
Hashes are now calculated from filenames in your index.
"From disk" folder data and name variants have been added to calculate hashes with filenames from disk.
This version also fixes an issue with folderdataandnames* searches (was previously searching folder data properties without names)
Added folder name hash properties.
These properties only calculate hashes on filenames.
Hashes are now calculated from filenames in your index.
"From disk" folder data and name variants have been added to calculate hashes with filenames from disk.
This version also fixes an issue with folderdataandnames* searches (was previously searching folder data properties without names)
Added folder name hash properties.
These properties only calculate hashes on filenames.