On Mon, 11 Dec 2006, Nik Conwell wrote:
On Dec 11, 2006, at 12:29 PM, Andrew Morgan wrote:
On Mon, 11 Dec 2006, Nik Conwell wrote:
I'm using the UW mailutil to transfer mailboxes from UW to Cyrus (2.3.7).
It uses APPEND, specifically multiappend (single APPEND with multiple
messages being appended). Cyrus-imapd handles this multiappend by
creating stage files for each appended message and leaving the file
descriptor open. The problem is that after 240 messages, we run out of
file descriptors and so an open() of the next stage file fails with
EMFILE. I updated /etc/cyrus.conf to make the max fds be 1024 (AFAICT
kernel MAX) which helped somewhat but not for larger mailboxes with > 1008
messages.
Shouldn't the multiappend/append be closing the FD for each stage file and
then reopening it later as it needs it?
Do people just tweak their kernels to have some insane number of FDs
available in order to compensate for this?
Or, do people not use mailutil and instead use something that issues
multiple append commands rather than a single append with multiple
e-mails?
We run with a much, much larger number of file descriptors here. I've
increased the system limit to around 200k (/proc/sys/fs/file-max on linux).
This is for the day-to-day running of Cyrus, so I don't know if you would
need a higher limit for running mailutil (but I doubt it).
In practice, each of my backends has only used a maximum of around 12k file
descriptors, but I'd hate to run out! :)
That's for the entire system though, right? I'm running into a 1024 limit
per process, namely the cyrus imap server process has all the appended stage
files open. Am I missing something fundamental here? (I probably am because
I would have figured people would have run into this issue already...)
BTW - my /proc/sys/fs/file-max has 406572.
Ah, then you just need to up the ulimits in your cyrus init script.
Something like:
# Crank up the limits
ulimit -n 209702
ulimit -u 2048
ulimit -c 102400
Andy
----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
