performance tuning

[Joe Hrbek]

Hi everyone!

I've been trying to tweak my cyrus system to improve performance.  I have a
few programs (smtp-sink that came with postfix) that I use to test my
server.  I used smtp-sink to send 100 messages to a random set of 20
different accounts (2,000 messages total; average time to send is about 60
sec).  That part works great, or well enough at least. :)  It takes about 2
seconds to send 100.  Not as fast as a lot of the other system specs I have
seen out there, but I'm using mysql for virtual users and a ton of expensive
filtering lookups in my mta, so it takes that long to shove them into the
queue manager.  Anyway, I've noticed that postfix can only deliver so many
messages at a time to lmtp.  I'm using lmtp as the delivery method, going
over a local tcp/ip transport (I was using lmtp:unix files previously, no
significant difference in performance between the two).  After all of the
mail is injected into the postfix queue, it takes about 30 seconds to
deliver all 2,000 into the correct mailboxes.  Really not that bad imho, but
I'm wondering if it could be better. :)  My goal is to get cyrus to keep up
with postfix, if that's possible.

I've tried all sorts of things, mostly blind guesses though.  I switched
from lmtp:unix to straight lmtp tcp/ip, that didn't make a difference at
all.  I also disabled "duplicatesuppression" in imapd.conf at the expense of
losing sieve (cringe), but that didn't make a significant difference either.
It took about 40 seconds to deliver 2000 messages with duplicatesuppression
enabled, about 8-10 seconds longer than when duplicatesuppression was
disabled.  Also, a postfix related control, I have
lmtp_destination/recipient_concurrency_limit set to 300.  I have expanded
this as high as 1000 but with not impact, which leads me to believe there
must be something I can do with cyrus to get it to accept more mail at one
time.

As a side note, when I run my programs, it brings my dual xeon 2.4ghz
processors to 80% utilization (2 gigs of ram and raid 0). :D  Not important
to the problem I don't think, but there ya go anyway, just if you are
curious.

Any ideas?  Let me know if I left something out.

Cyrus IMAP4 v2.1.16
postfix 2.0.16
mysql 4.x (allows 1000 concurrent connections)

Thanks!