idled vs poll

[Adam Stephens]

Ken Murchison wrote:
> Jorey Bump wrote:
>> Cyrus IMAP advertises IDLE, and the docs mention that the default method is 
>> "poll", but shed little light on the use of idled as an alternative. Which 
>> method is preferred, better performing (for users and/or servers),   and 
>> more robust?
> 
> idled will provide the client with instant updates, where poll will only give 
> them at the given interval.  I don't know if idled has been tested under a 
> heavy load.
>

We deployed idled here over the summer on our main staff IMAP server, a Sunfire 
v480 running Solaris 8 and cyrus-imap 2.2.12. It typically has about 13,000 
concurrent IMAP processes, most of which are not using idle as our main desktop 
client doesn't support it.

Initially, we saw a small rise in the server load, which wasn't 
particularly worrying, and occasionally slightly more worrying sluggish 
authentication. Last week was the start of term, and the system 
collapsed completely. The load average rose from about 1 to between 6 
and 9, and socket communication started failing, breaking LMTP delivery, 
SASL authentication and IDLE, all of which communicate over file system 
sockets:

cannot connect to saslauthd server: Connection refused
Failed to connect to socket /var/cyrus/imap/socket/lmtp for local_cyrus_deliver
transport: Connection refused
error sending to idled: 0

This left the system almost completely unusable. We tried various things to fix 
it, to no avail. Last night we rebuilt Cyrus with with-idle=poll; the load is 
immediately much lower (currently 0.7), authentication and IMAP are vastly more 
responsive and there are no socket errors logged.

Incidentally, our main student server, running Solaris 9, showed no such 
problems. It's unclear if this is due to the different usage patterns (the 
students, generally speaking, make much less intensive use of the email system, 
and aren't all back yet) or if it's related to the different OS version. 
The former seems more likely.

cheers,

Adam Stephens.
-- 
--------------------------------
Adam Stephens
Network Specialist - Email & DNS
adam.stephens at bristol.ac.uk