Re: Help please, cyrus imapd production system down

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



> Sometime during the night something happend to my imap store about (23:19
> CST). I have executed 'ctl_mboxlist -u < $backup' from two days ago to
> recover the mboxlist and have done a 'reconstruct -r user.$USER' for a few
> users to get their $USER.seen folders working again. The process lmtpunix
> is aborting, I cannot receive email (sendmail to deliver). I can still
> send mail out (through sendmail) so that part's working.
>
> I'm concerned that I need to recover my primary imap databases.
>
> Can someone help me diagnose what's going on and how to fix it (and what
> happened)?
>
> I am a small site with less than 30 email accounts in cyrus.
>
> There were no cyrus, sendmail, nor other email changes yesterday.

Hm, the 2.3.1 package you use may not be the most stable. Anyway, I guess
you have checked the basic things like full filesystems and so. Then, what
I suggest is

stop cyrus-imapd
mv /var/lib/imap/deliver.db /var/lib/imap/deliver.db.old
mv /var/lib/imap/db /var/lib/imap/db.old
mv /var/lib/imap/db.backup1 /var/lib/imap/db.backup1.old
mv /var/lib/imap/db.backup2 /var/lib/imap/db.backup2.old
do a full offline reconstruct and start cyrus again

Simon

>
> Mike
>
> Fedora Core 5 (current patches)
> $ rpm -qa | grep cyrus
> cyrus-imapd-2.3.1-2.8.fc5
> cyrus-imapd-devel-2.3.1-2.8.fc5
> cyrus-imapd-utils-2.3.1-2.8.fc5
> cyrus-sasl-2.1.21-10
> cyrus-sasl-devel-2.1.21-10
> cyrus-sasl-ldap-2.1.21-10
> cyrus-sasl-lib-2.1.21-10
>
> $ rpm -qa | grep db
> db4-4.3.29-8.fc5
> db4-devel-4.3.29-8.fc5
> db4-utils-4.3.29-8.fc5
> dbus-0.62-0.2.fc5
> dbus-devel-0.62-0.2.fc5
> dbus-glib-0.62-0.2.fc5
> dbus-python-0.62-0.2.fc5
> dbus-sharp-0.62-0.2.fc5
> dbus-x11-0.62-0.2.fc5
> dhcdbd-1.15-2.FC5
> gdb-6.3.0.0-1.134.fc5
> gdbm-1.8.0-26.2
> gdbm-devel-1.8.0-26.2
> jakarta-commons-dbcp-1.2.1-3jpp_3fc
> libdbi-0.8.1-1.2.1
> libdbi-dbd-mysql-0.8.1a-1.2.1
> libdbi-drivers-0.8.1a-1.2.1
> mysql-connector-odbc-3.51.12-1.2.1
> nss_db-2.2-35
>
> entries from /var/log/maillog
> Mar 26 08:59:23 elo master[20695]: service lmtpunix pid 23909 in READY
> state: terminated abnormally
> Mar 26 08:59:23 elo master[23979]: about to exec
> /usr/lib/cyrus-imapd/lmtpd
> Mar 26 08:59:23 elo lmtpunix[23979]: executed
> Mar 26 08:59:24 elo sendmail[22334]: m2QDfxl0022334: ruleset=check_mail,
> arg1=<hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx>, relay=localhost [127.0.
> 0.1], reject=451 4.1.8 Domain of sender address
> hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx does not resolve
> Mar 26 08:59:24 elo sm-msp-queue[21753]: m2LNUjr6004989: to=$USER@$DOMAIN,
> delay=4+14:28:39, xdelay=00:00:02, mailer=relay, pri=97
> 52409, relay=[127.0.0.1], dsn=4.0.0, stat=Deferred: 451 4.1.8 Domain of
> sender address hlmbpzwby@xxxxxxxxxxxxxxxxxxxxx does not resolve
> Mar 26 08:59:25 elo pop3[23212]: accepted connection
> Mar 26 08:59:25 elo master[23980]: about to exec
> /usr/lib/cyrus-imapd/pop3d
> Mar 26 08:59:25 elo pop3[23980]: executed
> Mar 26 08:59:25 elo pop3[23212]: login: $MACHINE.$DOMAIN [10.1.2.77] $USER
> plaintext User logged in
> Mar 26 08:59:29 elo imap[20839]: SQUAT failed to open index file
> Mar 26 08:59:29 elo imap[20839]: SQUAT failed
> Mar 26 08:59:29 elo sendmail[23958]: m2QDxBv3023958:
> from=<IOR@xxxxxxxxxxxxx>, size=6096, class=0, nrcpts=1,
> msgid=<E1JeW9V-00018C-OB@ce
> valvorto41.vauntstrand.com>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN
> [10.1.2.1]
> Mar 26 08:59:30 elo sendmail[23984]: m2QDxBv3023958: to=<$USER@$DOMAIN>,
> delay=00:00:18, xdelay=00:00:00, mailer=cyrusv2, pri=126
> 096, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with
> localhost
> Mar 26 08:59:32 elo master[20695]: process 23955 exited, signaled to death
> by 7
> Mar 26 08:59:32 elo master[20695]: service lmtpunix pid 23955 in READY
> state: terminated abnormally
> Mar 26 08:59:32 elo master[23988]: about to exec
> /usr/lib/cyrus-imapd/lmtpd
> Mar 26 08:59:32 elo lmtpunix[23988]: executed
> Mar 26 08:59:32 elo sendmail[23957]: m2QDxACR023957:
> from=<uliedtke@xxxxxxxxxxxx>, size=8641, class=0, nrcpts=1,
> msgid=<151501c88f49$94a
> d7780$7c475ec8@Nichole>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN
> [10.1.2.1]
> Mar 26 08:59:32 elo sendmail[23959]: m2QDxDQY023959:
> from=<NoelleselectmanGoldsmith@xxxxxxxxxxxxxxx>, size=6314, class=0,
> nrcpts=1, msgi
> d=11fa701c88f49$97279f40$57e39cca@SN2564177124, proto=SMTP, daemon=MTA,
> relay=$DMZ.$DOMAIN [10.1.2.1]
> Mar 26 08:59:33 elo sendmail[23990]: m2QDxACR023957:
> to=<fmpcaqfc@$DOMAIN>, delay=00:00:19, xdelay=00:00:00, mailer=cyrusv2,
> pri=1
> 28641, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out
> with localhost
> Mar 26 08:59:33 elo sendmail[23992]: m2QDxDQY023959: to=<$USER@$DOMAIN>,
> delay=00:00:18, xdelay=00:00:00, mailer=cyrusv2, pri=1263
> 14, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with
> localhost
> Mar 26 08:59:33 elo sendmail[23938]: m2QDwxuZ023938:
> from=<n@xxxxxxxxxxxxx>, size=11014, class=0, nrcpts=1,
> msgid=<000601c88f49$04b7aef1
> $413589ac@hmwrugyt>, proto=ESMTP, daemon=MTA, relay=$DMZ.$DOMAIN
> [10.1.2.1]
> Mar 26 08:59:34 elo sendmail[23994]: m2QDwxuZ023938: to=<$USER@$DOMAIN>,
> delay=00:00:27, xdelay=00:00:00, mailer=cyrusv2, pri=13
> 1014, relay=localhost, dsn=4.4.2, stat=Deferred: Connection timed out with
> localhost
> Mar 26 08:59:40 elo sendmail[23967]: m2QDxJCu023967:
> from=<uliedtke@xxxxxxxxxxxx>, size=7569, class=0, nrcpts=1,
> msgid=<154301c88f49$98c
> 41130$7c475ec8@Nichole>, proto=SMTP, daemon=MTA, relay=$DMZ.$DOMAIN
> [10.1.2.1]
> Mar 26 08:59:40 elo master[20695]: process 23916 exited, signaled to death
> by 7
> Mar 26 08:59:40 elo master[20695]: service lmtpunix pid 23916 in READY
> state: terminated abnormally
> Mar 26 08:59:40 elo master[23998]: about to exec
> /usr/lib/cyrus-imapd/lmtpd
> Mar 26 08:59:40 elo lmtpunix[23998]: executed
> Mar 26 08:59:40 elo sendmail[23997]: m2QDxJCu023967: to=$USER,
> delay=00:00:19, xdelay=00:00:00, mailer=cyrusv2, pri=37785, relay=local
>
>
>
> $ db_verify deliver.db  2>&1 | head
> db_verify: deliver.db: unexpected file type or format
> db_verify: Page 0: pgno incorrectly set to 1953720684
> db_verify: Page 0: bad magic number 543582572
> db_verify: Page 0: bad page size 16777216
> db_verify: Page 1: bad page number 1206496269
> db_verify: Page 1: bad page type 255
> db_verify: Page 1: unknown page type 255
> db_verify: Page 2: bad page number 1701145449
> db_verify: Page 2: bad page type 142
> db_verify: Page 2: unknown page type 142
> ......
> db_verify: Page 228: bad page type 109
> db_verify: Page 228: unknown page type 109
> db_verify: Page 229: bad page number 1465276490
> db_verify: Page 229: bad page type 105
> db_verify: Page 229: unknown page type 105
> db_verify: deliver.db: DB_VERIFY_BAD: Database verification failed
>
>
> $ file *.db
> annotations.db:  Cyrus skiplist DB
> deliver.db:      Cyrus skiplist DB
> mailboxes.db:    Cyrus skiplist DB
> tls_sessions.db: Cyrus skiplist DB
>
>
> $ file *.db
> annotations.db:  Cyrus skiplist DB
> deliver.db:      Cyrus skiplist DB
> mailboxes.db:    Cyrus skiplist DB
> tls_sessions.db: Cyrus skiplist DB
> [cyrus@elo ~]$ cat /etc/cyrus.conf
> # standard standalone server implementation
>
> START {
>   # do not delete this entry!
>   recover       cmd="ctl_cyrusdb -r"
>
>   # this is only necessary if using idled for IMAP IDLE
>   idled         cmd="idled"
> }
>
> # UNIX sockets start with a slash and are put into /var/lib/imap/sockets
> SERVICES {
>   # add or remove based on preferences
>   imap          cmd="imapd" listen="imap" prefork=10
>   imaps         cmd="imapd -s" listen="imaps" prefork=5
>   pop3          cmd="pop3d" listen="pop3" prefork=2
>   pop3s         cmd="pop3d -s" listen="pop3s" prefork=2
>   sieve         cmd="timsieved" listen="sieve" prefork=10
>
>   # these are only necessary if receiving/exporting usenet via NNTP
> #  nntp         cmd="nntpd" listen="nntp" prefork=3
> #  nntps                cmd="nntpd -s" listen="nntps" prefork=1
>
>   # at least one LMTP is required for delivery
> #  lmtp         cmd="lmtpd" listen="localhost:lmtp" prefork=1
>   lmtpunix      cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=10
>
>   # this is only necessary if using notifications
> #  notify       cmd="notifyd" listen="/var/lib/imap/socket/notify"
> proto="udp" prefork=1
> }
>
> EVENTS {
>   # this is required
>   checkpoint    cmd="ctl_cyrusdb -c" period=30
>
>   # this is only necessary if using duplicate delivery suppression,
>   # Sieve or NNTP
>   delprune      cmd="cyr_expire -E 3 -X 1" at=0200
>
>   # this is only necessary if caching TLS sessions
>   tlsprune      cmd="tls_prune" at=0400
>
>   # mikee 080109 - make sure messages are trimmed
>   msgprune1     cmd="ipurge -f -d 2 -X user.%.backup" at=2200
>   msgprune2     cmd="ipurge -f -d 5 -X user.%.spam" at=2230
>   msgprune3     cmd="ipurge -f -d 4 -X user.%.Trash" at=2300
> }
>
>
> $ cat /etc/imapd.conf
> admins: cyrus
> allowanonymouslogin: no
> allowplaintext: yes
> autocreatequota: -1
> autocreateinboxfolders: Sent | Drafts | | spam | backup
> configdirectory: /var/lib/imap
> drachost: localhost
> dracinterval: 0
> hashimapspool: true
> partition-default: /var/spool/imap
> poptimeout: 10
> postmaster: postmaster
> quotawarn: 90
> reject8bit: no
> sasl_mech_list: PLAIN
> sasl_pwcheck_method: saslauthd
> sendmail: /usr/lib/sendmail
> servername: servername
> sieveusehomedir: 0
> sievedir: /var/lib/imap/sieve
> sieve_maxscripts: 5
> sieve_maxscriptsize: 96
> timeout: 1440
> tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
> tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd-071024.pem
> tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd-071024.pem
> #unixhierarchysep: yes
>
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>


----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

[Index of Archives]     [Cyrus SASL]     [Squirrel Mail]     [Asterisk PBX]     [Video For Linux]     [Photo]     [Yosemite News]     [gtk]     [KDE]     [Gimp on Windows]     [Steve's Art]

  Powered by Linux