On 05.12.2013 22:46, Chuck Munro wrote:
>> On 04.12.2013 14:05, John Doe wrote:
>>>> From: Lists<lists@xxxxxxxxxxxxxxxxxx>
>>>>
>>>>>> Our next big test is to try out ZFS filesystem send/receive in
>>>>>> lieu
>>>>>> of
>>>>>> our current backup processes based on rsync. Rsync is a fabulous
>>>>>> tool,
>>>>>> but is beginning to show performance/scalability issues dealing
>>>>>> with
>>>>>> the
>>>>>> many millions of files being backed up, and we're hoping that ZFS
>>>>>> filesystem replication solves this.
>>>>
>>>> Not sure if I already mentioned it but maybe have a look at:
>>>> ?http://code.google.com/p/lsyncd/
>> I'm not so sure inotify works well with millions of files, not to
>> mention it uses rsync. :D
>>
>> -- Sent from the Delta quadrant using Borg technology! Nux!
>
> I can attest to the usefulness of 'lsyncd' for large numbers of files
> (our file server has almost 2 million in active use, with a second
> backup server that's lsync'd to the first.
>
> Things to note:
> - Yes, lsyncd does use rsync, but it issues an 'exclude *' followed by
> the list of only the file(s) that need updating at that moment.
>
> - The inotify service can be jacked waaaay up (three kernel
> parameters)
> to handle millions of files if you wish. Just make sure you have lots
> of RAM.
Be careful with it. Sadly I found out that inotify would consistently
fail on InnoDB files (ibd); I had to use stupid while loops and check
mtimes to perform some stuff that inotify-cron would've done much more
elegantly ...
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
