On Tue, 29 Aug 2006 09:51:49 +0100 (BST), "David Carter" <dpc22@xxxxxxxxx> said: > On Sun, 27 Aug 2006, Bron Gondwana wrote: > > > To tell you the truth, I'm seriously considering writing a replacement to > > sync_client that does a bunch of different things including multiple > > replicas, maintaining log files, etc. All of this drops out pretty easily > > from a pattern which produces a single log file per day and calls the > > sync_client fork children with a byte-range on the log file to run rather > > than moving the file and then running the copy. > > I think that you would be better off with multiple log files and multiple > sync_client processes, one for each replica. That way each replication > stream is independent and can progress at its own best speed. > > Particularly important if a replica dies (or is shut down for routine > maintenance) and needs to catch up from a big backlog of transactions. I fail to see the difference between this and a single big log file where each replica keeps track of where it's up to in the log file and then seeks to that spot, reads up to say 10,000 lines and processes that, then stores the new location so they next process can seek directly there before starting. This pattern means a lot less copying data to multiple log files and deleting again. It certainly doesn't mean every replica needs to be at the same point in the log file! This is how mysql handles replication - with each replica keep track of two values - which log file it's in (they're numbered sequentially starting from 1, and going up to wherever you get to before you reset replication for some reason) and how many bytes through that file it is. Bron. -- Bron Gondwana brong@xxxxxxxxxxx ---- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
