On Sat, Jun 30, 2018 at 7:10 PM Dave Chinner <david@xxxxxxxxxxxxx> wrote: > > On Fri, Jun 29, 2018 at 09:37:27PM -0500, Steve French wrote: > > I have been looking at i/o patterns from various copy tools on Linux, > > and it is pretty discouraging - I am hoping that I am forgetting an > > important one that someone can point me to ... > > > > Some general problems: > > 1) if source and target on the same file system it would be nice to > > call the copy_file_range syscall (AFAIK only test tools call that), > > although in some cases at least cp can do it for --reflink > > copy_file_range() should be made to do the right thing in as many > scnearios as we can document, and then switch userspace over to use > it at all times. Aggregate all the knowledge in one place, where we > know what the filesystem implementations are and can get hints to do > the right thing. Strongly agree. Looking at the code for what cp and rsync are trying to in userspace - and doing it without all of the information available in the VFS and fs layer and trying to do it across platforms - it is never going to be as good as what the kernel could do. On the otherhand, even in userspace, robocopy (among other Windows tools) seems to have sane options and do a much better job of parallelizing, but makes more sense in kernel. > > 5) sparse file support > > (and it would also be nice to support copy_file_range syscall ... but > > that is unrelated to the above) > > make copy_file_range() handle that properly. > > You also forgot all the benfits we'd get from parallelising > recursive directory walks and stat()ing inodes along the way (i.e. > the dir walk that rsync does to work out what it needs to copy). Very good point > > Am I missing some magic tool? Seems like Windows has various options > > for copy tools - but looking at Linux i/o patterns from these tools > > was pretty depressing - I am hoping that there are other choices. > > Not that I know of. The Linux IO tools need to dragged kicking and > screaming out of the 1980s. :/ Sure looks that way -- Thanks, Steve
