On Tue, 21 May 2013, Łukasz Michalski wrote:
Hi,
I am running cyrus imapd 2.4.11 on linux machine.
Today I had a deadlock involving cyr_expire and imapd process.
imapd was locked on (strace):
fcntl64(17, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}^C
<unfinished ...>
where fd=17 is a user index file (lsof):
imapd 32314 cyrus 17u REG 8,3 30944 10462461
/var/spool/imap/domain/c/cenbench.pl/a/user/arek^dydo/cyrus.index
Unfortunetaly I did not check cyr_expire with strace, but lsof showed this:
cyr_expir 24356 cyrus 0u CHR 1,3 0t0 509 /dev/null
cyr_expir 24356 cyrus 1u CHR 1,3 0t0 509 /dev/null
cyr_expir 24356 cyrus 2u CHR 1,3 0t0 509 /dev/null
cyr_expir 24356 cyrus 3u CHR 1,3 0t0 509 /dev/null
cyr_expir 24356 cyrus 4u CHR 1,3 0t0 509 /dev/null
cyr_expir 24356 cyrus 5u REG 8,2 144 19196113
/var/lib/imap/annotations.db
cyr_expir 24356 cyrus 6u REG 8,2 13300 18911268
/var/lib/imap/mailboxes.db
cyr_expir 24356 cyrus 7r FIFO 0,5 0t0 5678136 pipe
cyr_expir 24356 cyrus 8w FIFO 0,5 0t0 5678136 pipe
cyr_expir 24356 cyrus 9r FIFO 0,5 0t0 5678137 pipe
cyr_expir 24356 cyrus 10w FIFO 0,5 0t0 5678137 pipe
cyr_expir 24356 cyrus 11u REG 8,2 171032 19196126
/var/lib/imap/deliver.db
cyr_expir 24356 cyrus 12uR REG 8,2 0 26961663
/var/lib/imap/lock/domain/c/cenbench.pl/a/user/arek^dydo.lock
cyr_expir 24356 cyrus 13u REG 8,3 30944 10462461
/var/spool/imap/domain/c/cenbench.pl/a/user/arek^dydo/cyrus.index
There was 50 imapd processes (my upper limit) in locked on the same file
and a single cyr_expire. After killing cyr_expire I had to manually kill
all imapd processes to allow master to spawn new ones.
Not that my cyrus works on really, really slow machine. It is VM running
under KVM with I/O access varying from 5 to 60MB/s (as shown by hdparm -t)
Please let me know what can I do to trace it better next time.
Regards,
Łukasz
You probably will want to run reconstruct on that user's mailbox because
the cyrus.index file may be corrupted at this time. Also, you should
upgrade to Cyrus v2.4.17 if you can. There have been a large number of
bugfixes since your version, one of which may be the cause of your
deadlock.
Andy
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
