IMAP processes out of control
shashaness at hotmail.com
Tue Sep 22 19:12:55 EDT 2015
> On Sep 22, 2015, at 2:17 PM, Andrew Morgan <morgan at orst.edu> wrote:
> On Tue, 22 Sep 2015, Shaheen Bakhtiar wrote:
>> It happened again….. although it took longer for it to happen, this has been happening only since the upgrade in Jun.
>> The number of imap processes continues to increase until the server is completely OOM. the increase is drastic and all of a sudden.
> You should probably set maxchild to a value that won't run your server out of memory. :)
> Have you looked at the processes to see what they have in common? For example, sometimes an IMAP client will run amok and make hundreds or thousands of connections. Or perhaps the processes are all stuck waiting on a lock, etc.
> lsof, strace, netstat, and your Cyrus logs can help a lot.
[shawn at postoffice ~]$ ps aux | grep imapd | wc -l
[shawn at postoffice ~]# ps aux | grep imapds | wc -l
[shawn at postoffice ~]# ps aux | grep pop3d | wc -l
[shawn at postoffice ~]# ps aux | grep timseived | wc -l
[shawn at postoffice ~]# ps aux | grep lmtpunix | wc -l
Based on that output I changed the configuration file (below) adding maxchild. Most likely all my users have their clients open, and from previous monitoring I average about 200 instances of imapd:
# standard standalone server implementation
# do not delete this entry!
recover cmd="ctl_cyrusdb -r"
# this is only necessary if using idled for IMAP IDLE
# UNIX sockets start with a slash and are put into /var/lib/imap/sockets
# add or remove based on preferences
imap cmd="imapd" listen="imap" prefork=5 maxchild=300
imaps cmd="imapd -s" listen="imaps" prefork=1 maxchild=100
pop3 cmd="pop3d" listen="pop3" prefork=3 maxchild=5
pop3s cmd="pop3d -s" listen="pop3s" prefork=1 maxchild=5
sieve cmd="timsieved" listen="sieve" prefork=0
# these are only necessary if receiving/exporting usenet via NNTP
# nntp cmd="nntpd" listen="nntp" prefork=3
# nntps cmd="nntpd -s" listen="nntps" prefork=1
# at least one LMTP is required for delivery
# lmtp cmd="lmtpd" listen="lmtp" prefork=0
lmtpunix cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1
# this is only necessary if using notifications
# notify cmd="notifyd" listen="/var/lib/imap/socket/notify" proto="udp" prefork=1
# this is required
checkpoint cmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprune cmd="cyr_expire -E 3" at=0400
# this is only necessary if caching TLS sessions
tlsprune cmd="tls_prune" at=0400
Comments, concerns??? I’m going to keep observium open on a separate screen and watch when it starts going up, than do the lsof,netstat, and watch logs to see if I can tell why the sudden upticks.
More information about the Info-cyrus