We have a RHEL4u7 on all 5 servers: 1 mupdate master: mumailmaster 2 backends: mumailstore01, mumailstore02 2 Postfix MTA/Cyrus proxy frontends: mumail01, mumail02 So I started getting this on my backends around 14:15 EST, at which time mail started getting deferred to the backends. I have 20,000+ per frontend deferred for delivery at the time of this e-mail. Nov 19 16:30:26 mumailstore01 ctl_cyrusdb[4672]: DBERROR db4: PANIC: Cannot allocate memory Nov 19 16:30:26 mumailstore01 ctl_cyrusdb[4672]: DBERROR: critical database situation Nov 19 16:30:26 mumailstore01 ctl_mboxlist[4673]: DBERROR db4: PANIC: fatal region error detected; run recovery Nov 19 16:30:26 mumailstore01 ctl_mboxlist[4673]: DBERROR: critical database situation Nov 19 16:30:26 mumailstore01 ctl_cyrusdb[4674]: DBERROR db4: PANIC: fatal region error detected; run recovery Nov 19 16:30:26 mumailstore01 ctl_cyrusdb[4674]: DBERROR: critical database situation I tried the following directions for recovery. http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=skiplist&msg=32337. I made backup copies of all files deleted, renaming them $filename.corrupt. I did this on each server, recovered on the backends, and let it push the updates to the mupdate server. Manually recovered mailboxes.db on the frontends, as they did not seem to be getting updated. If I am going about this wrong, please someone point me in the right direction for documentation on murder disaster recovery. The following, while somewhat helpful, does not go into a great amount of detail: http://cyrusimap.web.cmu.edu/imapd/install-murder.html. So at this point my user agent says Unknown/Invalid partition. The partitions are correctly defined on the backend mail stores. A 'ctl_mboxlist -d' shows correct partitions, no matter which local mailboxes.db I attempt to dump. Furthermore, LMTP is still not delivering to the backends during this outage. Any helpful tips or pointers would be appreciated. Thanks, -- Eric G. Wolfe, IT Associate, Sr. One John Marshall Drive Marshall University, Drinko Library 428k Huntington, WV 25755 Phone: 304.696.3428 Email: eric.wolfe@xxxxxxxxxxxx "Who is General Failure and why is he reading my hard disk ?" Microsoft spel chekar vor sail, worgs grate !! (By leitner@xxxxxxxxxxxxxxxx, Felix von Leitner) ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html