Quickest way to delete many small files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 06/12/2013 10:26 AM, Pete Smith wrote:
> Hi
>
> We've got a directory with approx 1 million files. Mac user ... grr
>
> Is there a quick and dirty way to remove them?
>
> I was going with:
>
> ls | xargs rm
>
> but maybe rsync -a --delete empty/ full/
>
> would be less expensive?
>
> This is on a distributed replicated volume.


http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html

tl;dr The rsync method is likely to be faster because of readdir batching.  In 
fact, the results from that article are likely to be even more applicable to 
any distributed filesystem where readdir is even more expensive.  This 
difference is completely orthogonal to the difference between deleting files 
through a client mountpoint or directly on the brick, with or without renaming 
first.


[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux