imapd mem exhausted

[Patrick Welche]

On Wed, Sep 24, 2003 at 10:48:39AM -0400, Scott Adkins wrote:
> What version of Cyrus are you using?

CVS HEAD from yesterday, which is called 2.1.15.

>  We are using 2.2b1 here, and are
> experiencing something similar with memory issues for IMAP.  I brought up
> the discussion a few weeks ago about it and it was characterized as some
> kind of mmap() weirdness on our Tru64 platform, which I don't really think
> is the case.
> 
> I will look in our logs to see if we are seeing a similar issue with a
> lot of accepted connections as you are.

I actually only had 1 connection - I should have made it clear that those
were *all* the accepts in imapd.log (from my ... it looks as though the
line is repeated - sorry) - it was just to show that nothing
much was going on apart from that 1 imapd running for about 35mins until
that one imapd exhausted virtual memory.

> My experiences are as follows with the problem:
> 
>  1) It seems that the RSS footprint of the process increases when a user
>     makes a connection (from observation).

Yes - and for me, essentially the more messages I append, the greater
the increase. It doesn't matter which mailbox, nor whether the selected
mailbox changes. It seems to be a per message increase.

>  2) Most of the time, we may see a process jump up to 26 or 27MB in size
>     (and lately, we are now seeing them jump up to 30-32MB in size), but
>     typically go back down to several hundred KB shortly thereafter.  If
>     the system starts to slow down for whatever reason, the time for a
>     process to shrink increases, and we have more processes at the bigger
>     RSS size and run the risk of exhausting memory and swap.
> 
>  3) The problem was characterized as an mmap() problem on Tru64 because
>     our mailboxes.db file is about 27MB in size.  However, we are seeing
>     the sizes jump to 30-32MB in size, and our mailboxes.db file is still
>     only about 27MB in size.  *shrugs*

Hmm, I'm using NetBSD, and my mailboxes.db is tiny (20k). Remember, I was
just testing with 1 user. There are 2 users total in the system.

>  4) Restarting the Cyrus server has a dramatic effect on memory usage by
>     putting all the processes back at the base several hundred KB size.
>     It takes quite a while before we see the issue #2 become a problem
>     again (even on a busy server).
> 
> Anyways, I have been looking at this problem over that last several days
> and my gut feeling has been leaning towards the on-connection route...
> which certainly jives with your message below...

For me it seems there is a memory leak in append :/ Maybe for you each
imap connection receives another few messages => they grow? So memory isn't
even freed between connections?

Cheers,

Patrick