Performance problems, compared to wu-imapd

[Kendrick Vargas]

On Thu, 29 Jan 2004, Tuomas Toropainen wrote:

> The problem is that wu-imapd seems to perform much better than cyrus.
> Cyrus version is 2.1.16, compiled with default database options. Syslog is
> not logging debug messages, and has synchronous logging disabled (-). Both
> systems have 500 test users, about 100 mails per user in INBOX. Mails are
> generated with postal and are random text, max 20kB size. I also tried
> with 100 users, the results are much same. Mstone results, squirrelmail
> speed and computer load all tell that wu-imapd is faster.
>
> For example, with 500 users, using cyrus mstone managed to retrieve 8,730
> messages during 5 minute test, averaging 652.70ms per message. But using
> wu-imapd, 33,709 messages were retrieved, averaging 15.71ms per message.
> Maximum concurrent connections was a little lower with wu-imapd (~180 vs
> 220), but with cyrus computer load increased to 150 when there were 190
> imapd processes. Wu-imapd caused a maximum load of ~15.
>
> What am I doing wrong, or is this computer just too slow for cyrus to
> perform well? Or are the default databases so poor that they cayse this?

The difference isn't so much the number of users as it is the number of
messages. The difference between a standard UW-IMAP and Cyrus install are
that UW-IMAP stores mail in a mbox format and cyrus stores it in something
similar to (but more advanced than) a maildir format.

With mbox, all the messages are stored in a single file. If your messages
were all 20k or less, that means your hundred message mailboxes were a
maximum 2 megabytes in size. As we all know, 2 meg isn't anything to get
excited about. So whenever you hit uw-imap, it was loading your 2 meg file
completely into memory and thus all operations were basically hitting ram.
All the daemon had to really worry about was the file being appended to
while it was open.

With cyrus' mailbox format, all of the messages are stored as individual
files. There's an index to help make things faster, however whenever you
try to retrieve a full message you are creating disk activity. The file
has to be open and spat back at you. With cyrus the mailbox isn't forced
to be opened completely in memory. Instead of one 2 megabyte file, you had
100 20 kilobyte files.

The REAL difference in your testing would have been seen if you'd had
individual mailboxes with hundreds to thousands of messages (even tens of
thousands) with messages ranging in size from 2k to 300k or hell, even up
to a meg or two. If you'd tried this with uw-imap, you would've QUICKLY
seen the performance drop, especially with inboxes nearing the size of
your available ram and with a ton of concurrent users. This is all because
of the way uw-imap opens it's inboxes.

While cyrus may be slower in some instances than uw-imap, it's gonna be a
helluva lot more consistent. While in the scenario in the last paragraph,
uw-imap's performance would've suffered, cyrus performance likely would've
stayed pretty much the same. This, of course, is due to the way cyrus
stores it's messages. If you configured sendmail and uw-imap to store it's
messages in maildir format (which I believe you can), then you'd have a
real test for testing the performance of one over the other. But at that
point, the configuration time to change the way it works makes the change
useless. May as well go with cyrus :-)

Let's also not forget the extra features of cyrus. Sieve (server side
email filtering); Murder (multi-machine distributable mailbox stores);
partitioning so that you can put users in different areas of your
filesystem depending on where you have space; home directory agnostic mail
storage for stuff typically outside the INBOX. Hell, users don't even have
to exist on the system for cyrus to accept mail for them. All this makes
cyrus great for enterprise style applications, which is something you
won't get from uw-imap no matter how hard you squeeze.
			-peace

-- 
Let he who is without clue kiss my ass