Cyrus 2.5.9 imapd children and tcp_keepalive & idle socket questions

[Andy Dorman]

On 09/13/2016 02:15 AM, Bron Gondwana via Info-cyrus wrote:
> On Tue, 13 Sep 2016, at 10:44, ellie timoney via Info-cyrus wrote:
>> Note that it talks about the process being used for new connections.
>> Each imapd process serves one connection at a time (so if you have 50
>> client connections, you will have 50 imapd processes to serve them, plus
>> whatever you have preforked ready to serve new connections).  When one
>> of these connections disconnects, the imapd process that was serving
>> that connection goes back into the pool, ready to serve a new incoming
>> connection -- unless it has already served $uses connections, in which
>> case it exits, and master spawns a new one instead if necessary.
>>
>> So you can see how, if you have mostly long-lived connections, each
>> imapd's count of how many connections it has served would grow very very
>> slowly -- many may not ever reach their limit, because you end up
>> needing to reboot the server (OS security updates, data centre works,
>> etc) before that occurs.
>
> If you look in /var/lib/imap/proc/* you should see one file for each process id,
> which tells you who is connected to that process, and which folder they have
> selected (if any).  This is quite useful for tracking down if there's a single user
> causing issues.
>
> Bron.
>
>

Bron, that was incredibly useful (FWIW, the proc list in Debian is at 
/run/cyrus/proc/)

I took a look and our worst case server appears to have two users 
(actually one user with two separate addresses) that are causing the 
trouble. I restarted cyrus-imapd about 10 hrs ago and we are up to 28 
imapd processes with start times all through the 10-hr block and they 
all appear to be owned by one of two addresses as you can see here:

proc/10470:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/10473:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/11077:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/11257:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/11760:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/11810:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/12320:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/13065:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/13374:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/13376:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/13656:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/13764:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/13922:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/13944:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/14326:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/14506:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/17323:imap  b2b at cogift.co       cogift.co!user.b2b        Idle
proc/18363:imap  donations at cogift.co cogift.co!user.donations  Idle
proc/8701:imap   b2b at cogift.co       cogift.co!user.b2b        Idle
proc/8705:imap   donations at cogift.co cogift.co!user.donations  Idle
proc/8771:imap   donations at cogift.co cogift.co!user.donations  Idle
proc/892:imap    b2b at cogift.co       cogift.co!user.b2b        Idle
proc/8960:imap   b2b at cogift.co       cogift.co!user.b2b        Idle
proc/9144:imap   donations at cogift.co cogift.co!user.donations  Idle
proc/9197:imap   b2b at cogift.co       cogift.co!user.b2b        Idle
proc/943:imap    donations at cogift.co cogift.co!user.donations  Idle
proc/9584:imap   b2b at cogift.co       cogift.co!user.b2b        Idle
proc/9800:imap   donations at cogift.co cogift.co!user.donations  Idle

So, given that all of these report as idle, should they not be shutting 
down at some point?

I have the settings below in imapd.conf. My understanding of them is 
after a connection has been idle for 15 min, try 5 tcp probes at 75 sec 
intervals before marking the connection as broken.

tcp_keepalive: 1
tcp_keepalive_idle: 900
tcp_keepalive_cnt: 5
tcp_keepalive_intvl: 75

So is it possible that this client is responding to the tcp keepalive 
packets so the idle connection is never shut down, but the client keeps 
opening new connections when it wants to check email (which appears to 
happen about every 10 - 15 minutes)?

I know this user so I will ask her what email client she is using and 
how often it is set to check email.

Also, I see this note in cyrus.conf about idled...

# this is only necessary if idlemethod is set to "idled" in imapd.conf
idle		cmd="idled"

But I can't find anything about idlemethod in any of the cyrus docs or 
man pages.  All I can find to set in imapd.conf for idle is the socket:

idlesocket: /var/run/cyrus/socket/idle

Hmmmmm...wait a minute...in looking for the Debian proc directory I came 
across another idle socket created when I restarted cyrus-imapd.

srwxrwxrwx 1 root root 0 Sep 14 06:10 /run/cyrus/socket/idle

However, our imapd.conf specified a different idle socket which was also 
created when cyrus-imapd was restarted.

srwxrwxrwx 1 root root 0 Sep 14 06:10 /var/run/cyrus/socket/idle

So I am thinking I should modify my imapd.conf idlesocket to 
"/run/cyrus/socket/idle" which cyrus seems to be setting up anyway.  Or 
am I completely missing an obvious point here?

Sorry for being so clueless and thanks for all the help.

-- 
Andy Dorman
Ironic Design, Inc.
AnteSpam.com