Cyrus Imap Benchmark

[Bron Gondwana]

On Sun, 14 Sep 2008 22:00:32 -0700 (PDT), "faris" <farislinux at yahoo.com> said:
>  
> Hi All,
>  
> I have a special requirement and need to benchmark cyrus imap with 100K
> mailboxes having sasld with mysql authentication. the benchmark will be
> writing mails with 400KB size attachement to all 100K mailboxes, while
> reading all the 100k mailboxes simultanously, each mailbox have 10MB
> quota allocated.
>  
> Following is the configuration of the server which i am planning to use
> for the loadtest.
>  
> Server Conf: Intel(R) Xeon(TM) CPU 3.80GHz X 4,

That's an awful lot of CPU.

> 4GB ram

That's an awfully small amount of RAM

(our biggest machines have 32GB RAM and a _single_ CPU.  They're not CPU
 limited)

> OS: Red Hat Enterprise Linux Server release 5.1
> cyrus-sasl-2.1.22-4
> cyrus-imapd-2.3.7-1.1.el5

that's getting old.  In particular, it has skiplist bugs, and you'll be
wanting skiplist for your mailboxes.db.  Can you upgrade it?

> pam_mysql-0.7-0.4.rc1.el5.2
> MySQL-server-4.0.24-0

I take back my "getting old" statement!  That's ancient.  Then again,
unlikely to be an issue.

> postfix-2.3.11-1.rhel5

I see an awfully big lack of information about your disk configuration here,
given that IO is going to be your limiting factor.  Care about that above
the other specs.  Seriously.

> Can anybody tell me what is a maximum amount of mailboxes cyrus-imapd
> supports? also is it possbile for me to go upto 200K mailboxes in a
> single node ? Are there any tools available to do the benchmark for this
> scanario ? 

We've had 200k mailboxes.  It's pushing sanity a bit.  I assume you mean
200k users, rather than 200k folders.

We have that many on a single machine, but we don't keep them on a single
instance these days, mainly because restoring that much disk after a
filesystem failure takes too long.  Instead, we split everything into
300Gb instances.  There may be up to 30 or so of them on a single machine
if the disk arrays are big enough.

Then again, we have in the past.  Cyrus coped fine.  You'll be wanting
that extra RAM and a 64bit kernel though, so you can actually cache the
hundreds of thousands of cyrus.index files.  Also, you'll want to have
split meta and meta files (cyrus.header, cyrus.index, cyrus.cache) on
really fast drives.  We use 10k or 15k RPM drives in RAID1 for meta.

Regards,

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm