Cyrus 2.5.9 imapd children and tcp_timeout questions
Nicola Nye
nicolan at fastmail.com
Mon Sep 12 17:02:59 EDT 2016
Hi Andy,
I remember writing a faq about process counts growing.
https://cyrusimap.org/imap/faqs/o-toomanyprocesses.html
I'm not actually sure it's applicable in your situation, but just
in case...
Cheers,
Nicola
----- Original message -----
From: "Andy Dorman via Info-cyrus" <info-cyrus at lists.andrew.cmu.edu>
To: info-cyrus at lists.andrew.cmu.edu
Subject: Re: Cyrus 2.5.9 imapd children and tcp_timeout questions
Date: Mon, 12 Sep 2016 07:40:28 -0500
Eliie, thank you for that information about imapd usage. I have only
been managing our imap for a few months and am still learning.
We do not have a lot of active users on each server (probably under 20),
but enough that what you describe could be happening.
As for how I am determining they are reaching their usage limit, all I
have are the process start times. When we hit the 100 limit I can see
that the preforked 5 are still running as are many that were started
many hours ago. I could be crazy wrong (probably am ;-) but it seemed to
me that if processes were being killed and recycled I should not see a
lot of old processes sitting around.
And I owe an apology to Konrad...I originally wrote this email a week
ago and then set it aside while I tried setting tcp_timeout=1. Then
when that did not help, I sent the email and forgot to update my
imapd.conf. This is what I have in my imapd.conf.
tcp_keepalive: 1
tcp_keepalive_idle: 900
tcp_keepalive_cnt: 5
tcp_keepalive_intvl: 75
It would be great if anyone who is using tcp_keepalive could share their
settings in case I completely misunderstood how these work.
In the meantime I am going to bump my max processes up to 200 and my
monitoring warning level to 150 see how that works.
Thanks for all the help everyone.
Andy
On 09/11/2016 11:05 PM, ellie timoney via Info-cyrus wrote:
>> We now have 4 of the 14 servers where the number of imapd processes
>> slowly grows and usually within 12-14 hours reaches the imapd process
>> limit (currently set to 100).
>
> How many user accounts are on these servers? Each client connection
> constitutes one imapd process on the server.
>
> It's my understanding that Thunderbird (for example) in its default
> configuration holds 5 connections open at once, and tries to keep them
> open (either with IDLE, if enabled, or NOOP), and will reconnect them
> after a while if they disconnect. (I would expect most other IMAP
> clients of a similar maturity to behave similarly.) So it would only
> take 20 users leaving their mail client open most of the time to hit
> your limit of 100 concurrent imapd processes.
>
> So 100 seems like a very small number, unless you have a very small
> number of user accounts.
>
>> From what we can tell watching the
>> process list, the old processes are never shut down after
>> reaching their
>> usage limit (which is the default).
>
> Do you have more information about how you determined that their usage
> limit had been reached?
>
>> While we had this problem intermittently with 2.4.18 (once or twice a
>> month on a random server), it is much worse since the update
>> to 2.5.9.
>
> This might simply be 2.5.9 doing a better job of helping clients keep
> open connections open?
>
> Cheers,
>
> ellie
>
> On Mon, Sep 12, 2016, at 01:01 PM, Andy Dorman via Info-cyrus wrote:
>> Hi everyone. We have 14 Debian servers running the current Debian
>> release of Cyrus imap 2.5.9.
>>
>> We upgraded from 2.4.18 to 2.5.9 on Aug 28, upgraded the cyrus.conf &
>> imapd.conf to rename the deprecated options and switch from
>> skiplist to
>> twoskip for all our dbs. Our cyrus.conf and imapd.con (minus
>> comments)
>> is at the end of this email.
>>
>> We now have 4 of the 14 servers where the number of imapd processes
>> slowly grows and usually within 12-14 hours reaches the imapd process
>> limit (currently set to 100). From what we can tell watching the
>> process list, the old processes are never shut down after
>> reaching their
>> usage limit (which is the default).
>>
>> The difference between the 4 with the problem and the others
>> have to be
>> the mail clients. Each server supports a different group of mail
>> domains and users, so I suspect some of the users on the problem
>> servers
>> are using one or more mail clients that are not well behaved,
>> but I have
>> no indication of what they are doing wrong.
>>
>> While we had this problem intermittently with 2.4.18 (once or twice a
>> month on a random server), it is much worse since the update
>> to 2.5.9.
>> We now have to restart imapd on all the problem servers almost
>> twice a
>> day.
>>
>> If it matters, we have tried both idled and polling and have not seen
>> any difference.
>>
>> So has anyone else seen an issue like this? Does anyone have any
>> suggestions on how to debug?
>>
>> We have examined syslog and do not see any strange reports outside of
>> the normal berkeley DBERROR messages we always see due to bdb
>> compiled
>> into the debian Cyrus release.
>>
>> ===== cyrus.conf =====
>> START {
>> recover cmd="/usr/sbin/cyrus ctl_cyrusdb -r"
>> idle cmd="idled"
>> delprune cmd="/usr/sbin/cyrus expire -E 3"
>> tlsprune cmd="/usr/sbin/cyrus tls_prune"
>> }
>>
>> SERVICES {
>> imap cmd="imapd" listen="*:imap" prefork=5 maxchild=100
>> imaps cmd="imapd -s" listen="*:imaps" prefork=5 maxchild=100
>> lmtp cmd="lmtpd" listen="*:lmtp" prefork=5 maxchild=20
>> sieve cmd="timsieved" listen="*:sieve" prefork=0 maxchild=100
>> }
>>
>> EVENTS {
>> checkpoint cmd="/usr/sbin/cyrus ctl_cyrusdb -c" period=30
>> delprune cmd="/usr/sbin/cyrus expire -E 3" period=120
>> tlsprune cmd="/usr/sbin/cyrus tls_prune" period=60
>> resquat cmd="/usr/sbin/cyrus squatter -s -r -i *" at=0437
>> }
>>
>> ===== imapd.conf =====
>> admins: cyrus
>>
>> allowallsubscribe: on
>>
>> allowanonymouslogin: no
>>
>> allowplaintext: on
>>
>> altnamespace: on
>>
>> annotation_db: twoskip
>>
>> anyoneuseracl: off
>>
>> autocreate_quota: 512000
>>
>> configdirectory: /var/lib/cyrus
>>
>> defaultdomain: ironicdesign.com
>>
>> defaultpartition: default
>>
>> duplicate_db: twoskip
>>
>> expunge_mode: delayed
>>
>> fulldirhash: on
>>
>> guid_mode: sha1
>>
>> hashimapspool: on
>>
>> idlesocket: /var/run/cyrus/socket/idle
>>
>> improved_mboxlist_sort: on
>>
>> lmtp_downcase_rcpt: on
>>
>> lmtpsocket: /var/run/cyrus/socket/lmtp
>>
>> mboxname_lockpath: /run/cyrus/lock
>>
>> notifysocket: /var/run/cyrus/socket/notify
>>
>> partition-default: /var/spool/cyrus/mail
>>
>> popminpoll: 1
>>
>> proc_path: /run/cyrus/proc
>>
>> quota_db: twoskip
>>
>> sasl_log_level: 7
>> sasl_mech_list: PLAIN
>> sasl_pwcheck_method: saslauthd
>> sasl_minimum_layer: 0
>> allowapop: no
>>
>> sieveusehomedir: false
>> sievedir: /var/spool/cyrus/sieve
>>
>> statuscache_db: twoskip
>>
>> syslog_prefix: cyrus
>>
>> tls_sessions_db: twoskip
>>
>> tls_client_ca_file: /etc/ssl/certs/mail.ironicdesign.com.pem
>>
>> tls_client_ca_dir: /etc/ssl/certs
>>
>> # The rest of our TLS setup
>> tls_server_cert: /etc/ssl/certs/mail.ironicdesign.com.pem
>> tls_server_key: /etc/ssl/certs/mail.ironicdesign.com.pem
>> tls_session_timeout: 1440
>>
>> tls_ciphers: TLSv1+HIGH:!aNULL:@STRENGTH
>>
>> tls_versions: tls1_0 tls1_1 tls1_2
>>
>> umask: 077
>>
>> unix_group_enable: off
>>
>> unixhierarchysep: on
>>
>> userdeny_db: twoskip
>>
>> virtdomains: userid
>>
>> sieve_extensions: fileinto reject vacation imapflags notify envelope
>> body relational regex subaddress copy
>> ===== end imapd.conf =====
>>
>> Thanks for any ideas on what to look for or how to debug this.
>>
>> --
>> Andy Dorman
>> Ironic Design, Inc.
>> AnteSpam.com
>>
>> ----
>> Cyrus Home Page: http://www.cyrusimap.org/
>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
>> To Unsubscribe:
>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20160913/1351aab4/attachment.html>
More information about the Info-cyrus
mailing list