Re: rsync question: building list taking forever

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



>
> > Don't forget that the time taken to build the file list is a function of
> > the number of files present, and not their size. If you have many
> millions
> > of small files, it will indeed take a very long time. Over sshfs with
> > a slowish link, it could be days.
> >
> > ....and it may end up failing silently or noisily anyway.


Ahhh, but isn't that part of the beauty of adventure that being a linux
admin is all about? *twitch*

On Sun, Oct 19, 2014 at 8:27 PM, Cliff Pratt <enkiduonthenet@xxxxxxxxx>
wrote:

> On Mon, Oct 20, 2014 at 7:57 AM, Steve Thompson <smt@xxxxxxxxxxxx> wrote:
>
> > On Sun, 19 Oct 2014, Keith Keller wrote:
> >
> >  I suspect that sshfs's relatively poor performance is having an impact
> >> on your transfer.  I have a 30TB filesystem which I rsync over an
> >> OpenVPN link, and building the file list doesn't take that long (maybe
> >> an hour?).  (The links themselves are reasonably fast; if yours are not
> >> that would have a negative impact too.)
> >>
> >
> > Don't forget that the time taken to build the file list is a function of
> > the number of files present, and not their size. If you have many
> millions
> > of small files, it will indeed take a very long time. Over sshfs with
> > a slowish link, it could be days.
> >
> > ....and it may end up failing silently or noisily anyway.
>
> Cheers,
>
> Cliff
> _______________________________________________
> CentOS mailing list
> CentOS@xxxxxxxxxx
> http://lists.centos.org/mailman/listinfo/centos
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos




[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]
  Powered by Linux