Best app to find / clean / sort duplicate files

#1

I am bringing back files from different sources on one drive

Along the year there were backup of backup of backup etc

What good app do you know to search all files on a drive and help cleaning up the mess with duplicates ?

Something that could allow me to say : if you find duplicate, keep the most recent file.

#2

FSlint without hesitation, it makes a 3 verification of any file, with MD5SUM and other two signatures that I doesn't remember and shows you all duplicates even with different names.

You find it at Elive Synaptic tool.

The problem is that if you have a lot of files it take several hours to analyse them and it has no persistence, if you reboot you must analyse again.

The other problem is that you can automatically clean duplicate files with some criteria but until you are very familiarized with FSlint I suggested you to manually delete your files that will take you a lot of time, but its safer.

I goes slow, and every week I run FSlint to clean my tons of duplicate files. It will take me several months but I have less risk to delete accidentally files that I want to keep (some files I want to have them duplicates in several folders, and FSlint in automatised detection will delete all except one, so I prefer do it manually)

Try it, it's an amazing tool. FSlint + Unison is the best way to store your files as best as possible, since I discovered both my life has changed. :boogie::boogie::boogie:

2 Likes
#3

tks " a lot " for your reply !
Much appreciated..

:happybounce:

1 Like
#4

@Max
I brought all my Data on one disk
I ran Fslint and I have 40 GB of duplicates

I will google and read but is there a feature, didn't find it yet, to just say " go go keep one copy of each files " ?

I am asking that as the list is very long ....

#5

OK I found a short tutorial by a very motivate person LOL

JF

#6

Hi @yoda I came into home now, sorry.

I suppose with YT video you finally found the feature to remove ALL duplicates EXCEPT one (the oldest, the youngest, etc...) did you?

If not, I'll take a look to show you where is that feature.

#7

Yes I found but thks

I am just running it again before saying " Go clean up " and will close my eyes ( joke)

#8

Good luck, but normally you mustn't be afraid, FSlint will delete all your duplicate files except one :coding::excited::runaway::surprise::love:

#9

I am surprise it can even match the same video files with different names !

#10

Yes, as I told you FSlint check MD5SUM and other two signatures for each file so it will find all duplicate files you have even if they have different names. You'll love it, and you have only 40 GB of duplicate files :wink:

1 Like
#11

OOPS 40 GB saving was the 1st pass.. Thought I was done but

I forgot another USB 1 TB drive LOL

Now bringing it's content on the 4 TB one and will run again Fslint

once done, part time, I'll have to sort / place files in the right folders / create a good hierarchy for my folders

#12

LOL :laugh:

Once you'll delete all duplicates it's effectively much more easy to range your unique files in a clean and logical hierarchy... take your time.... don't try to do all in one day or you'll finish fool.... :omfg:

1 Like
#13

ok just saved a little 121 GB :wink:

tks @maxinou !!

saved close to 200 GB and " a lot of time"

1 Like
#14

Happy that I got you service :dance:

And a lot of "you're welcome" :smile:

1 Like
#15

Thanks so much for this hint,
(never heard about it before, as a 'payback' - I do recommend "Aptik" to you :wink: )

= now I can finally try to sort the damn photo collection [9000 pics all in all or so ...]
especially the numerous backups from my wife -
I guess the pics are there for more than 5x and each is in HDR (6-12MB !) from her Samsung Galxy S9+
(women & technical stuff = :omfg: :face_vomiting: :hot_face::nauseated_face: :sniff: )

Fortunately it is also available in the repos of my main-production machine, look:

It's scanning now - will tell about the result not before tomorrow, I guess..... :music: :popcorn:

1 Like
#16

massively-select them in thunar, right click, "resize images"
select "auto"
they will be resized in a 2/3 of their resolution and the size will be much smaller too!
it will ask if to replace the original ones or create a temporal dir of resized images

same for mp3 songs, shrinking features! :w00t:

thats an own feature of elive :slight_smile:

@Rebel450 which package manager is that? looks pretty friendly :slight_smile:

3 Likes
#17

Thanx for the hint, didn't know this! :omfg: :w00t:

= :disbelief: well...... :hot_face: ... not only ... :fearful::cold_sweat:

it is (or better: was..) .. the ubuntu default - now Mint devs has forked it and call it nowadays:
mintinstall%20vintage

But it has changed meanwhile and looks like this now:

#18

I wonder if it could be forked to be used in elive, it should be completely debian compatible & standard

we should know the name of the package or executable, and see if we have it in debian available too

if im not wrong, there's other package managers available too for elive

didnt like the second screenshot :confused: looks a bit commercial thing :thinking:

1 Like
#19

AAAAAAAAAAAAAAAAAAAAAAAAAAA = :omfg: :eyepopping: :news: :cold_face: :broken_heart: :facepunch:

LOOK:

Message:
116,428,398,592 bytes wasted in 36935 files (in 21014 groups)

W T F !? 116.43GB wasted = :face_vomiting: :imp: :bomb: :fist_right: :facepunch: :woman_red_haired:

(no, will not slap her , but may be ... ) :slight_smile: LOL

1 Like
#20

we should switch to calamares better - it's also for you much better configurable
and comfortable to maintain (as to run after / clean obsolet becoming depencies every now and then) !