I'm trying to find out if anyone else sees intermittent skiplist recovery
problems as sometimes we do, usually after a cyrus restart.
Our particular setup is cyrus 2.3.6, but we've seen this problem with
everything from 2.1 onwards. This is on on x86 with linux 2.6.x, various
versions. We use skiplists for the seen state db, and that's where the
problem always occurs.
Basically it seems that after restarting cyrus, sometimes none, sometimes a
couple, no more than 2 or 3 of the user seen state databases will be
corrupted. Cyrus will report something along the lines of the following in
the log:
Jun 13 18:31:00 imapx imap[21178]: DBERROR: skiplist recovery: 01F8 should
be INORDER
Jun 13 18:31:00 imapx imap[21178]: DBERROR: opening
/var/cyrus/imapx/user/x/xxx.seen: cyrusdb error
Jun 13 18:31:00 imapx imap[21178]: DBERROR: skiplist recovery: 01F8 should
be INORDER
Jun 13 18:31:00 imapx imap[21178]: DBERROR: opening
/var/cyrus/imapx/user/x/xxx.seen: cyrusdb error
Now checking the mailing list, it appears we're not the only ones who have
seen this:
http://www.irbs.net/internet/info-cyrus/0507/0075.html
Doing as suggested there (truncating up to the problem point) does seem to
make the error go away, and at least the skiplist can be recovered and
written to again. Without that, the skiplist is effectively "dead", and no
new data can be written to it, annoying in itself. Once the error is fixed,
and if cyrus runs normally, then we don't see new corrupted seen state
databases forming. It seems to ONLY happen after a cyrus restart, so it
sounds like when an imapd shuts down due to a kill signal, it doesn't always
cleanup the skiplist properly or something like that.
What I wanted to do is get an idea of how common this is with people, since
speaking with Ken the other day he says that they've never seen this problem
in production on their systems. So I was hoping people who also see this
could report their cyrus version and OS + hardware config. Hopefully we can
hope to narrow this down a bit.
Rob
----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
