Bug or feature: (Too) Many imapd processes hanging around?

[Andreas Haumer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi!

I did some further investigation on this issue and could resolve it,
at least for now...

Am 04.10.2012 13:32, schrieb Andreas Haumer:
[...]
> 
> But since September 26th I see the number of imapd processes continuously growing. As of today there are more than 2000 of them:
> 
> ravel:~ # ps ax | grep imapd | wc -l 2057
> 
[...]
> 
> ravel:~ # ll /cluster/var/imap/user/o/office.seen -rw------- 1 cyrus mail 57804 27. Sep 14:11 /cluster/var/imap/user/o/office.seen
> 
> 
> I checked about 20 processes and all of them hang on the F_SETLKW fcntl() call on fileid #16 which always points to the same file (/cluster/var/imap/user/o/office.seen)
> 

Using lsof I could identify the process which was holding the
lock on /cluster/var/imap/user/o/office.seen:


ravel:/var/log # lsof /cluster/var/imap/user/o/office.seen | grep 16uW
imapd   27522 cyrus   16uW  REG  147,0    57804 3686853 /cluster/var/imap/user/o/office.seen


This process itself was waiting in a futex() call!

A SIGHUP did not help, but a SIGTERM made the process terminate:

ravel:~ # strace -p 27522
Process 27522 attached
futex(0x7f5c8b149620, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
- --- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=20435, si_uid=0} ---
rt_sigreturn()                          = 202
futex(0x7f5c8b149620, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
- --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=20435, si_uid=0} ---
+++ killed by SIGTERM +++

I could not find out what the futex call was waiting for (since
September 26th!), but after the process terminated, all other imapd
processes waiting for the lock continued and eventually terminated,
too.

Now the number of imapd processes is back to normal:

ravel:~ # ps ax | grep imapd | wc -l
38

(with currently about 15 active users)

It seems, everything is fine now.

But the question remains: why did this happen in the first place?
Is there a known bug (race condition?) somewhere in cyrus-imapd 2.3.18?
Is there some known configuration issue which could lead to this behaviour?

I would rather not upgrade to 2.4 bypassing the official OpenSUSE
packages (well, if absolutely necessary, I could, but I would like
to avoid this path)

Thanks!

- - andreas

- -- 
Andreas Haumer                     | mailto:andreas at xss.co.at
*x Software + Systeme              | http://www.xss.co.at/
Karmarschgasse 51/2/20             | Tel: +43-1-6060114-0
A-1100 Vienna, Austria             | Fax: +43-1-6060114-71
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iD8DBQFQbYPvxJmyeGcXPhERAmyIAJ9OfGewykr0qy6PzpHR8FxCBOCZiwCePYAY
PefxLrqNAKCkky5j47yTkVo=
=TGCZ
-----END PGP SIGNATURE-----