> Sometime during the night something happend to my imap store about (23:19 > CST). I have executed 'ctl_mboxlist -u < $backup' from two days ago to > recover the mboxlist and have done a 'reconstruct -r user.$USER' for a few > users to get their $USER.seen folders working again. The process lmtpunix > is aborting, I cannot receive email (sendmail to deliver). I can still > send mail out (through sendmail) so that part's working. > > I'm concerned that I need to recover my primary imap databases. > > Can someone help me diagnose what's going on and how to fix it (and what > happened)? > > I am a small site with less than 30 email accounts in cyrus. > > There were no cyrus, sendmail, nor other email changes yesterday. Hm, the 2.3.1 package you use may not be the most stable. Anyway, I guess you have checked the basic things like full filesystems and so. Then, what I suggest is stop cyrus-imapd mv /var/lib/imap/deliver.db /var/lib/imap/deliver.db.old mv /var/lib/imap/db /var/lib/imap/db.old mv /var/lib/imap/db.backup1 /var/lib/imap/db.backup1.old mv /var/lib/imap/db.backup2 /var/lib/imap/db.backup2.old do a full offline reconstruct and start cyrus again Simon > > Mike > > Fedora Core 5 (current patches) > $ rpm -qa | grep cyrus > cyrus-imapd-2.3.1-2.8.fc5 > cyrus-imapd-devel-2.3.1-2.8.fc5 > cyrus-imapd-utils-2.3.1-2.8.fc5 > cyrus-sasl-2.1.21-10 > cyrus-sasl-devel-2.1.21-10 > cyrus-sasl-ldap-2.1.21-10 > cyrus-sasl-lib-2.1.21-10 > > $ rpm -qa | grep db > db4-4.3.29-8.fc5 > db4-devel-4.3.29-8.fc5 > db4-utils-4.3.29-8.fc5 > dbus-0.62-0.2.fc5 > dbus-devel-0.62-0.2.fc5 > dbus-glib-0.62-0.2.fc5 > dbus-python-0.62-0.2.fc5 > dbus-sharp-0.62-0.2.fc5 > dbus-x11-0.62-0.2.fc5 > dhcdbd-1.15-2.FC5 > gdb-6.3.0.0-1.134.fc5 > gdbm-1.8.0-26.2 > gdbm-devel-1.8.0-26.2 > jakarta-commons-dbcp-1.2.1-3jpp_3fc > libdbi-0.8.1-1.2.1 > libdbi-dbd-mysql-0.8.1a-1.2.1 > libdbi-drivers-0.8.1a-1.2.1 > mysql-connector-odbc-3.51.12-1.2.1 > nss_db-2.2-35 > > entries from /var/log/maillog > Mar 26 08:59:23 elo master[20695]: service lmtpunix pid 23909 in READY > state: terminated abnormally > Mar 26 08:59:23 elo master[23979]: about to exec > /usr/lib/cyrus-imapd/lmtpd > Mar 26 08:59:23 elo lmtpunix[23979]: executed > Mar 26 08:59:24 elo sendmail[22334]: m2QDfxl0022334: ruleset=check_mail, > arg1=<hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx>, relay=localhost [127.0. > 0.1], reject=451 4.1.8 Domain of sender address > hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx does not resolve > Mar 26 08:59:24 elo sm-msp-queue[21753]: m2LNUjr6004989: to=$USER@$DOMAIN, > delay=4+14:28:39, xdelay=00:00:02, mailer=relay, pri=97 > 52409, relay=[127.0.0.1], dsn=4.0.0, stat=Deferred: 451 4.1.8 Domain of > sender address hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx does not resolve > Mar 26 08:59:25 elo pop3[23212]: accepted connection > Mar 26 08:59:25 elo master[23980]: about to exec > /usr/lib/cyrus-imapd/pop3d > Mar 26 08:59:25 elo pop3[23980]: executed > Mar 26 08:59:25 elo pop3[23212]: login: $MACHINE.$DOMAIN [10.1.2.77] $USER > plaintext User logged in > Mar 26 08:59:29 elo imap[20839]: SQUAT failed to open index file > Mar 26 08:59:29 elo imap[20839]: SQUAT failed > Mar 26 08:59:29 elo sendmail[23958]: m2QDxBv3023958: > from=<IOR@xxxxxxxxxxxxx>, size=6096, class=0, nrcpts=1, > msgid=<E1JeW9V-00018C-OB@ce > valvorto41.vauntstrand.com>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN > [10.1.2.1] > Mar 26 08:59:30 elo sendmail[23984]: m2QDxBv3023958: to=<$USER@$DOMAIN>, > delay=00:00:18, xdelay=00:00:00, mailer=cyrusv2, pri=126 > 096, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with > localhost > Mar 26 08:59:32 elo master[20695]: process 23955 exited, signaled to death > by 7 > Mar 26 08:59:32 elo master[20695]: service lmtpunix pid 23955 in READY > state: terminated abnormally > Mar 26 08:59:32 elo master[23988]: about to exec > /usr/lib/cyrus-imapd/lmtpd > Mar 26 08:59:32 elo lmtpunix[23988]: executed > Mar 26 08:59:32 elo sendmail[23957]: m2QDxACR023957: > from=<uliedtke@xxxxxxxxxxxx>, size=8641, class=0, nrcpts=1, > msgid=<151501c88f49$94a > d7780$7c475ec8@Nichole>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN > [10.1.2.1] > Mar 26 08:59:32 elo sendmail[23959]: m2QDxDQY023959: > from=<NoelleselectmanGoldsmith@xxxxxxxxxxxxxxx>, size=6314, class=0, > nrcpts=1, msgi > d=11fa701c88f49$97279f40$57e39cca@SN2564177124, proto=SMTP, daemon=MTA, > relay=$DMZ.$DOMAIN [10.1.2.1] > Mar 26 08:59:33 elo sendmail[23990]: m2QDxACR023957: > to=<fmpcaqfc@$DOMAIN>, delay=00:00:19, xdelay=00:00:00, mailer=cyrusv2, > pri=1 > 28641, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out > with localhost > Mar 26 08:59:33 elo sendmail[23992]: m2QDxDQY023959: to=<$USER@$DOMAIN>, > delay=00:00:18, xdelay=00:00:00, mailer=cyrusv2, pri=1263 > 14, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with > localhost > Mar 26 08:59:33 elo sendmail[23938]: m2QDwxuZ023938: > from=<n@xxxxxxxxxxxxx>, size=11014, class=0, nrcpts=1, > msgid=<000601c88f49$04b7aef1 > $413589ac@hmwrugyt>, proto=ESMTP, daemon=MTA, relay=$DMZ.$DOMAIN > [10.1.2.1] > Mar 26 08:59:34 elo sendmail[23994]: m2QDwxuZ023938: to=<$USER@$DOMAIN>, > delay=00:00:27, xdelay=00:00:00, mailer=cyrusv2, pri=13 > 1014, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with > localhost > Mar 26 08:59:40 elo sendmail[23967]: m2QDxJCu023967: > from=<uliedtke@xxxxxxxxxxxx>, size=7569, class=0, nrcpts=1, > msgid=<154301c88f49$98c > 41130$7c475ec8@Nichole>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN > [10.1.2.1] > Mar 26 08:59:40 elo master[20695]: process 23916 exited, signaled to death > by 7 > Mar 26 08:59:40 elo master[20695]: service lmtpunix pid 23916 in READY > state: terminated abnormally > Mar 26 08:59:40 elo master[23998]: about to exec > /usr/lib/cyrus-imapd/lmtpd > Mar 26 08:59:40 elo lmtpunix[23998]: executed > Mar 26 08:59:40 elo sendmail[23997]: m2QDxJCu023967: to=$USER, > delay=00:00:19, xdelay=00:00:00, mailer=cyrusv2, pri=37785, relay=local > > > > $ db_verify deliver.db 2>&1 | head > db_verify: deliver.db: unexpected file type or format > db_verify: Page 0: pgno incorrectly set to 1953720684 > db_verify: Page 0: bad magic number 543582572 > db_verify: Page 0: bad page size 16777216 > db_verify: Page 1: bad page number 1206496269 > db_verify: Page 1: bad page type 255 > db_verify: Page 1: unknown page type 255 > db_verify: Page 2: bad page number 1701145449 > db_verify: Page 2: bad page type 142 > db_verify: Page 2: unknown page type 142 > ...... > db_verify: Page 228: bad page type 109 > db_verify: Page 228: unknown page type 109 > db_verify: Page 229: bad page number 1465276490 > db_verify: Page 229: bad page type 105 > db_verify: Page 229: unknown page type 105 > db_verify: deliver.db: DB_VERIFY_BAD: Database verification failed > > > $ file *.db > annotations.db: Cyrus skiplist DB > deliver.db: Cyrus skiplist DB > mailboxes.db: Cyrus skiplist DB > tls_sessions.db: Cyrus skiplist DB > > > $ file *.db > annotations.db: Cyrus skiplist DB > deliver.db: Cyrus skiplist DB > mailboxes.db: Cyrus skiplist DB > tls_sessions.db: Cyrus skiplist DB > [cyrus@elo ~]$ cat /etc/cyrus.conf > # standard standalone server implementation > > START { > # do not delete this entry! > recover cmd="ctl_cyrusdb -r" > > # this is only necessary if using idled for IMAP IDLE > idled cmd="idled" > } > > # UNIX sockets start with a slash and are put into /var/lib/imap/sockets > SERVICES { > # add or remove based on preferences > imap cmd="imapd" listen="imap" prefork=10 > imaps cmd="imapd -s" listen="imaps" prefork=5 > pop3 cmd="pop3d" listen="pop3" prefork=2 > pop3s cmd="pop3d -s" listen="pop3s" prefork=2 > sieve cmd="timsieved" listen="sieve" prefork=10 > > # these are only necessary if receiving/exporting usenet via NNTP > # nntp cmd="nntpd" listen="nntp" prefork=3 > # nntps cmd="nntpd -s" listen="nntps" prefork=1 > > # at least one LMTP is required for delivery > # lmtp cmd="lmtpd" listen="localhost:lmtp" prefork=1 > lmtpunix cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=10 > > # this is only necessary if using notifications > # notify cmd="notifyd" listen="/var/lib/imap/socket/notify" > proto="udp" prefork=1 > } > > EVENTS { > # this is required > checkpoint cmd="ctl_cyrusdb -c" period=30 > > # this is only necessary if using duplicate delivery suppression, > # Sieve or NNTP > delprune cmd="cyr_expire -E 3 -X 1" at=0200 > > # this is only necessary if caching TLS sessions > tlsprune cmd="tls_prune" at=0400 > > # mikee 080109 - make sure messages are trimmed > msgprune1 cmd="ipurge -f -d 2 -X user.%.backup" at=2200 > msgprune2 cmd="ipurge -f -d 5 -X user.%.spam" at=2230 > msgprune3 cmd="ipurge -f -d 4 -X user.%.Trash" at=2300 > } > > > $ cat /etc/imapd.conf > admins: cyrus > allowanonymouslogin: no > allowplaintext: yes > autocreatequota: -1 > autocreateinboxfolders: Sent | Drafts | | spam | backup > configdirectory: /var/lib/imap > drachost: localhost > dracinterval: 0 > hashimapspool: true > partition-default: /var/spool/imap > poptimeout: 10 > postmaster: postmaster > quotawarn: 90 > reject8bit: no > sasl_mech_list: PLAIN > sasl_pwcheck_method: saslauthd > sendmail: /usr/lib/sendmail > servername: servername > sieveusehomedir: 0 > sievedir: /var/lib/imap/sieve > sieve_maxscripts: 5 > sieve_maxscriptsize: 96 > timeout: 1440 > tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt > tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd-071024.pem > tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd-071024.pem > #unixhierarchysep: yes > > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html