Cyrus-imapd memory tuning

[lst_hoe02 at kwsoft.de]

Zitat von Marco <falon at ruparpiemonte.it>:

> Hello cyrus users,
>
>   I have a Cyrus-imapd server with 2400 mailboxes imap accessed by
> Open-Xchange client. Days ago this Cyrus-Imapd server experiences an
> Out of Memory and it starts to sacrifice childs:
>
>
> In average I have about 400 max simoultaneous connections and I have
> no problems on memory. I think a network issue (DNS or LDAP stalled)
> causes connections to suddenly increase to 3500. Imapd processes were
> opened until memory overflow.
>
> My server is:
> Red Hat Enterprise Linux Server release 6.3 (Santiago)
> Without problems I read something like this:
>
>               total       used       free     shared    buffers     cached
> Mem:       8061976    7651020     410956          0    1355964    3412788
> -/+ buffers/cache:    2882268    5179708
> Swap:      4194296      32180    4162116
>
> procs -----------memory---------- ---swap-- -----io---- --system--
> -----cpu-----
>   r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us
> sy id wa st
>   2  0  32180 386880 1356476 3423712    0    0   643   327   25   18
> 10  4 81  5  0
>
> current cyrus.conf:
> SERVICES {
>    # add or remove based on preferences
>    imap          cmd="imapd" listen="imap" prefork=5
>    pop3          cmd="pop3d" listen="pop3" prefork=3
>    sieve         cmd="timsieved" listen="sieve" prefork=0
>    lmtp          cmd="lmtpd -a" listen="lmtp" prefork=0
> }
>
>
>
> I have to prevent memory issue when some oddity forces clients to make
> DOS on Cyrus. So I would like to configure the maxchild cyrus
> parameter for imap. I would like to set this value to avoid memory
> issue during normal work, having a known value of system RAM.
>
> I see that an IMAPD process takes in average 22-25MB. With 8GB RAM,
> the server would swap already with less than 400 conns; it not
> happens, so this evaluation is wrong or too many conservative. I think
> that I better consider differences between RSS and SHR memory to
> tuning imapd processes number, but I'm not sure.

Hello,

not sure where you get the number of 22-25MB per imapd from, but to  
understand the real memory consumption per process one have to take  
care of the concepts of shared text, virtual memory and the like. A  
good estimate is to examine a process with "pmap -d <process-id>" and  
have a look at the writeable part. One one of my machine this is  
between 1M and 6M which is closer to the true value IMHO. Next problem  
is that you also have memory consumption around the process for  
network/file buffer, open files and the like which also adds to the  
memory pressure.

Some hints are here for example:  
http://www.ualberta.ca/CNS/RESEARCH/LinuxClusters/mem.html

So best would be to use pmap on a "typical" imapd on your system and  
add some margin for additional helper around to get a estimate how far  
you can get.

Regards

Andreas



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5958 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20140310/df084574/attachment-0001.bin