|
> There
may be an argument that could be made for 2 backup stratagies
That's the point. In the context of SME environments (Small and Medium-sized Enterprises, i.e. from 5 to 50 employees normally, up to 250 in some countries) that we were talking about, a replication is an overkill, IMO. But for large enterprises like MNCs, large universities, public mail providers (Fastmail) of course multiple masters and backups via replication is the way to go. For large deployments there are good backup solutions in Cyrus, but for the small businesses admins I don't know any to recommend. > If the mailboxes are on something like EXT4 you can do an LVM snapshot bacause of the built in auto checkpointing and for something like xfs there is freeze. Yepp, this is the idea. But the data on disk should be in a consistent state. Something like "FLUSH TABLES WITH READ LOCK" is what is needed actually, i.e. to consistently write to disk the data from the current instance and lock. > The trouble is that read operations can alter a files state so it may not be just a simple matter of a write lock. If you mean things like SEEN state that are changed when the user reads something, the implementation v1 could block it, v2 could allow it by queuing in memory such state-change pending operations. Anyway these are just implementation details that don't change the general logic and, taking into account the supposed duration of the lock, IMO don't even matter much. The significant work that blocks this feature is the global lock across the entire Cyrus. Once it's implemented, it would be much easier to introduced specific improvements here and there. > Cyrus has multiple databases that would also need to be frozen and flushed before the snapshot is taken. > If you spread your mailstorage and metadata storage over multiple file systems trying to co-ordinate snapshots becomes more complex. Sure, Cyrus would have to lock all the databases and where to store them would be up to the admin. Actually, fs snapshots is just the most obvious use-case. There could be others... up to the admin to decide. From:
Alvin Starr
Sent: Thursday, May 10, 2018 23:55 To: Info-cyrus Subject: Re: Backup methods Mysql provides table consistency by locking but that is only table consistency. Multiple updates across multiple tables could easily result in an inconsistent database even though the tables are individually consistent. With some tables that take hours to dump locking the tables is problematic. This is why some people use LVM snapshots combined with "FLUSH TABLES WITH READ LOCK". The trouble is that read operations can alter a files state so it may not be just a simple matter of a write lock. If the mailboxes are on something like EXT4 you can do an LVM snapshot bacause of the built in auto checkpointing and for something like xfs there is freeze. Cyrus has multiple databases that would also need to be frozen and flushed before the snapshot is taken. If you spread your mailstorage and metadata storage over multiple file systems trying to co-ordinate snapshots becomes more complex.
There may be an argument that could be made for 2 backup stratagies. 1) where the mailstoreage and metadata can exist on a single volume and flushing the various databases is a short duration event. Then an LVM snapshot could be used 2) for distribted large scale mail systems where only an online live backup system can be used. Backup for 100 users has different requirements than backup for 100000 users so why not support a few different backup strategies.
-- Alvin Starr || land: (905)513-7688 Netvel Inc. || Cell: (416)806-0133 alvin@xxxxxxxxxx || ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus |
---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
