On Dec 11, 2006, at 12:29 PM, Andrew Morgan wrote:
On Mon, 11 Dec 2006, Nik Conwell wrote:
I'm using the UW mailutil to transfer mailboxes from UW to Cyrus
(2.3.7). It uses APPEND, specifically multiappend (single APPEND
with multiple messages being appended). Cyrus-imapd handles this
multiappend by creating stage files for each appended message and
leaving the file descriptor open. The problem is that after 240
messages, we run out of file descriptors and so an open() of the
next stage file fails with EMFILE. I updated /etc/cyrus.conf to
make the max fds be 1024 (AFAICT kernel MAX) which helped somewhat
but not for larger mailboxes with > 1008 messages.
Shouldn't the multiappend/append be closing the FD for each stage
file and then reopening it later as it needs it?
Do people just tweak their kernels to have some insane number of
FDs available in order to compensate for this?
Or, do people not use mailutil and instead use something that
issues multiple append commands rather than a single append with
multiple e-mails?
We run with a much, much larger number of file descriptors here.
I've increased the system limit to around 200k (/proc/sys/fs/file-
max on linux). This is for the day-to-day running of Cyrus, so I
don't know if you would need a higher limit for running mailutil
(but I doubt it).
In practice, each of my backends has only used a maximum of around
12k file descriptors, but I'd hate to run out! :)
That's for the entire system though, right? I'm running into a 1024
limit per process, namely the cyrus imap server process has all the
appended stage files open. Am I missing something fundamental here?
(I probably am because I would have figured people would have run
into this issue already...)
BTW - my /proc/sys/fs/file-max has 406572.
----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
