How to remove large number of files on linux?

I had a client that left a broken script running for years and was wondering why his server was filling up.

They had multiple accounts with 500,000 to 1,000,000 files which were insane.  I started to remove with a find command but after 24 hrs hours later it was still running and barely put a dent in number of files.

find /home/user/mail/new -type f -exec rm -vf {} \;

I then found someone one talking in a forum about using rsync with delete.  So, next, I tried that out and wow, it was like 100x faster.

This is what i used:

rsync -av --delete /root/empty/ /home/user/mail/new

The rsync command removed all the files very fast, next issue is the directory it taking a larger number of bytes than the default 4096, so I also like to rename and remove the directory which the files were in.

drwxr-xr-x 2 user user 39071744 Aug 18 22:23 new/

after I delete and recreate, it should look like this:

drwxr-xr-x 2 user user 4096 Aug 19 09:00 new/

There was still a large number of inodes associated with the directory and I aways like to clear that out in addition to cleaning up all the files from a large directory.

I highly recommended this alternative to use rsync instead of using find and rm to clear out a large number of files in a directory.


Categories: Uncategorized