On Sun, Jul 24, 2016 at 02:38:10PM +0200, Arkadiusz Miśkiewicz wrote: > > Hello. > > I'm using rsnapshot to backup big servers (like 5TB fs, 25 000 000 inodes, > small files - mailboxes in form of maildirs, so each mail is a separate file). > Backup server - kernel 4.6.3, V4 xfs filesystems. What storage? What mount options? What is the xfs_info output? (/me points at http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F) > cp -al for that amount takes about 1.5 day. > rm -rf of hardlinked copy takes another 1.5 day So what's the bottleneck? Reading the directory structure/inodes into memory to make the copy? What's the IO performance look like? Are they CPU bound? What else is generating IO load at the same time? How big are the individual directories? How full is the filesystem? How many hardlinks in a "copy"? FWIW< you've got 25M inodes in the filesystem - how many hardlinks do you have in the filesystem? 100M? 200M? 1B? i.e. what's the scale of the directory structure that contains all the hard links? > (and toons of ram for these operations; causing OOM until recent kernels made > reclaim better, so no more OOM) What oom problems? Slabtop output during a test? > Now the weird part - similar operations on ext4 finish in matter of hours. So you probably need to identify where the difference in behaviour is - reading from disk, writing to disk, CPU usage, directory entry creation/removal speed, etc. > ps. Didn't do scientific comparison (I'm just viewing backup logs of two > similar mail servers (similar hardware, similar storage size) being backed up > to single backup server onto two partitions - one with xfs and one with ext4 > on it)) So the /destination/ files is either ext4 or XFS, but the source filesystem is the same? So how does "cp -al" work to create hardlinks when copying to a different filesystem? If this is a copy to a different filesystem, then it's a very different problem to "create/removing hardlinks are slow". Clearly I haven't understood what you are trying to describe, so can you please describe the problem in more detail and not assume I know anything about where you are copying from/to, what the hardware or filesystem layout is, etc. I know I haven't answered your question and just fired back a bunch of questions, but I need to know specifics to be able to have any chance of understanding the problem you are having. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
