All the patches mentioned here are available for download at: http://cyrus.brong.fastmail.fm/ We've been busy working on a bunch of cyrus issues since I last posted. In particular things we have hit cleaning up from the "sync_server left files lying around after a bail out that caused random messages to be overwritten a lot later" bug. Ken - some of these are definite candidates for upstream. As much as possible we've made new behaviour optional with the default being the current behaviour. REMAINING ON MY TODO LIST: ========================== * undoable "DELETE" * investigate why data path directories aren't always being created for domainsplit users but meta paths are on the replica. * use the MD5 based UUID and the RFC822.MD5 to do spot-checks in our checkreplication script, similarly RFC822.SIZE and RFC822.FILESIZE. * expand logging for message delivery to include the UUID and UID of the delivered message. WHAT'S NEW: =========== * Make UUIDs work at all. The initialisation order of the UUID subsystem was wrong, so we had very few messages with a non-zero UUID. Also, the documentation for provide_uuid was misleading/ difficult to understand. Make that clearer. * MD5 UUIDs - we've created a new scheme for UUID generation, of the format: 02[first 11 bytes of message file md5]. This allows some basic integrity checking of the file on disk, and is still plenty random. Also adds the non standard IMAP FETCHable items UUID, RFC822.MD5 (calculated on the fly), RFC822.FILESIZE (does a stat or looks at the MMAP result if something else needs it) * Abort sync on mismatched UUIDs (to go with above) - make failover much safer to avoid deleting files that got delivered while the other machine was the replica. * windows-1250/1251/1255 codepages - some users we having trouble searching for things in messages with these codepages. I believe this is already accepted upstream. * pcreposix - use the PCRE engine for regexp matching in sieve. The gnu posix library had some nasty edge case behaviour which we're hoping won't exist in the prce library. * sync_client permissions check - I thought this had gone in upstream but must have been missed. Ken, please apply this one, it's a really obvious typo fix! * Set internal date based on first Received header - this one is probably a lot more controversial - but we have a pile of messages all with an internaldate of November 2005 when we restored from backups and lost all internaldate information. We also have a pile with bogus values thanks to having had mtimes way out due to being on a replica. * Speaking of which, there's a patch to set the mtime of the file on the replica based on the internaldate as well! We try to keep all these values in sync so re-creating the internaldate correctly is much easier. * RENAME INBOX. It's a cow no matter how you look at it, broken in one way or another, but the thing that hurt us was that it broke replication. The patch we chose makes replication work again at the expense of losing all seen state for the copied messages. The alternative was much more invasive, because sync_server does the "rename" operation as the admin user and they don't get the special treatment. Man I hate special cases. Moo. * Statuscache. There has been a pretty major rewrite. It's a lot faster for the common case now, because it avoids statting the seen file, opting instead to make all codepaths that affect the seen file also delete the associated statuscache record. Faster reads, slightly slower writes for the seen DB. Guess what our users do more of. Regards, Bron. -- Bron Gondwana brong@xxxxxxxxxxx ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
