Performance problems, compared to wu-imapd

[Paul M Fleming]

What he said! UW-IMAP in sealed mode is supposed to perform fairly well,
but you don't get sieve, murder, partitions etc.. 

Having a sealed system in a non unix end-user environment is the way to
go. Our users have no idea what kind of server they are talking to --
just that it speaks IMAP ;) 


Kendrick Vargas wrote:
> 
> On Thu, 29 Jan 2004, Tuomas Toropainen wrote:
> 
> > The problem is that wu-imapd seems to perform much better than cyrus.
> > Cyrus version is 2.1.16, compiled with default database options. Syslog is
> > not logging debug messages, and has synchronous logging disabled (-). Both
> > systems have 500 test users, about 100 mails per user in INBOX. Mails are
> > generated with postal and are random text, max 20kB size. I also tried
> > with 100 users, the results are much same. Mstone results, squirrelmail
> > speed and computer load all tell that wu-imapd is faster.
> >
> > For example, with 500 users, using cyrus mstone managed to retrieve 8,730
> > messages during 5 minute test, averaging 652.70ms per message. But using
> > wu-imapd, 33,709 messages were retrieved, averaging 15.71ms per message.
> > Maximum concurrent connections was a little lower with wu-imapd (~180 vs
> > 220), but with cyrus computer load increased to 150 when there were 190
> > imapd processes. Wu-imapd caused a maximum load of ~15.
> >
> > What am I doing wrong, or is this computer just too slow for cyrus to
> > perform well? Or are the default databases so poor that they cayse this?
> 
> The difference isn't so much the number of users as it is the number of
> messages. The difference between a standard UW-IMAP and Cyrus install are
> that UW-IMAP stores mail in a mbox format and cyrus stores it in something
> similar to (but more advanced than) a maildir format.
> 
> With mbox, all the messages are stored in a single file. If your messages
> were all 20k or less, that means your hundred message mailboxes were a
> maximum 2 megabytes in size. As we all know, 2 meg isn't anything to get
> excited about. So whenever you hit uw-imap, it was loading your 2 meg file
> completely into memory and thus all operations were basically hitting ram.
> All the daemon had to really worry about was the file being appended to
> while it was open.
> 
> With cyrus' mailbox format, all of the messages are stored as individual
> files. There's an index to help make things faster, however whenever you
> try to retrieve a full message you are creating disk activity. The file
> has to be open and spat back at you. With cyrus the mailbox isn't forced
> to be opened completely in memory. Instead of one 2 megabyte file, you had
> 100 20 kilobyte files.
> 
> The REAL difference in your testing would have been seen if you'd had
> individual mailboxes with hundreds to thousands of messages (even tens of
> thousands) with messages ranging in size from 2k to 300k or hell, even up
> to a meg or two. If you'd tried this with uw-imap, you would've QUICKLY
> seen the performance drop, especially with inboxes nearing the size of
> your available ram and with a ton of concurrent users. This is all because
> of the way uw-imap opens it's inboxes.
> 
> While cyrus may be slower in some instances than uw-imap, it's gonna be a
> helluva lot more consistent. While in the scenario in the last paragraph,
> uw-imap's performance would've suffered, cyrus performance likely would've
> stayed pretty much the same. This, of course, is due to the way cyrus
> stores it's messages. If you configured sendmail and uw-imap to store it's
> messages in maildir format (which I believe you can), then you'd have a
> real test for testing the performance of one over the other. But at that
> point, the configuration time to change the way it works makes the change
> useless. May as well go with cyrus :-)
> 
> Let's also not forget the extra features of cyrus. Sieve (server side
> email filtering); Murder (multi-machine distributable mailbox stores);
> partitioning so that you can put users in different areas of your
> filesystem depending on where you have space; home directory agnostic mail
> storage for stuff typically outside the INBOX. Hell, users don't even have
> to exist on the system for cyrus to accept mail for them. All this makes
> cyrus great for enterprise style applications, which is something you
> won't get from uw-imap no matter how hard you squeeze.
>                         -peace
> 
> --
> Let he who is without clue kiss my ass