Remove Duplicate Files in Bash

Using the following command can remove duplicate files in current directory:

md5 * | sort -k 4 | uniq -f 3 -d | tr -d '()' | cut -d " " -f 2| xargs rm -v

md5: generate file checksums sort -k 4: sort output by column 4 uniq -f 3 -d: ignore the first 3 columns and only shows duplicated lines tr -d ‘()’: remove the ‘(’, ‘)’ characters so we can get duplicated file name by cut cut -d " " -f 2: using space as delimiter and get the file name we want to remove xargs rm -v: remove the duplicated files you may need to modify the command based on your md5 standard output. you may run it multiple times until it has no output