BUG? File descriptor use in cmd_append (for MULTIAPPEND) results
in many open files
morgan at orst.edu
Mon Dec 11 12:29:58 EST 2006
On Mon, 11 Dec 2006, Nik Conwell wrote:
> I'm using the UW mailutil to transfer mailboxes from UW to Cyrus (2.3.7). It
> uses APPEND, specifically multiappend (single APPEND with multiple messages
> being appended). Cyrus-imapd handles this multiappend by creating stage
> files for each appended message and leaving the file descriptor open. The
> problem is that after 240 messages, we run out of file descriptors and so an
> open() of the next stage file fails with EMFILE. I updated /etc/cyrus.conf
> to make the max fds be 1024 (AFAICT kernel MAX) which helped somewhat but not
> for larger mailboxes with > 1008 messages.
> Shouldn't the multiappend/append be closing the FD for each stage file and
> then reopening it later as it needs it?
> Do people just tweak their kernels to have some insane number of FDs
> available in order to compensate for this?
> Or, do people not use mailutil and instead use something that issues multiple
> append commands rather than a single append with multiple e-mails?
We run with a much, much larger number of file descriptors here. I've
increased the system limit to around 200k (/proc/sys/fs/file-max on
linux). This is for the day-to-day running of Cyrus, so I don't know if
you would need a higher limit for running mailutil (but I doubt it).
In practice, each of my backends has only used a maximum of around 12k
file descriptors, but I'd hate to run out! :)
More information about the Info-cyrus