Cyrus Patches used at FastMail.FM
robm at fastmail.fm
Tue May 23 03:47:04 EDT 2006
> Cool, some of the patches look really interesting and I'm considering to
> include one or the other into my rpm packages. For example the statuscache
> patch seems very nice. Just to be sure, are there any license restrictions
> on the patches?
No, no license restrictions. We'd love these to get back into cyrus itself,
so happy to contribute them back to whatever licence cyrus is under.
The statuscache is actually very useful, more so than Bron's comment on the
page immediately suggests. Originally I thought it was our webmail client
that was doing most of the status calls, so I actually put caching into our
web client. It turned out that in fact lots of email clients also blast the
IMAP server with status calls, so putting it in the server itself has made a
noticeable difference to load on our servers.
The other one that I've just finished today is the "fastindex" one, which
I'm also excited about. I discovered that on our brand new server with only
a couple of hundred users on it, cmdtimer was still showing that some status
calls were taking > 2 seconds for large mailboxes (for status calls that
weren't cached that is) on a virtually unloaded server (load < 0.05).
Checking it out, it became pretty obvious that the problem was the core
status loop which calls index_insequence() on every message to see if it's
seen/unseen. When the seen state sequence list becomes quite large, you're
doing an awful lot of repeated scans of the seen list.
With a bit of slightly messy hacking (static variables + extra parameter)
instead of cleaner but more major code surgery, I've added a "caching" mode
when scanning an assumed sequence list of messages into a sequential seen
list. This has reduced the status call to basically instant even on 40,000+
message folders with lots of random seen/unseen messages. I've put the
latest version of the patch up now:
> I'd love to see some of the patches integrated upstream, it makes my own
> work easier :)
So would we, the less we have to maintain, the better, and for large
installations with many IMAP users, the performance impact of the status and
fastindex patches should be helpful.
More information about the Info-cyrus