choosing a file system
robm at fastmail.fm
Sat Jan 3 22:56:59 EST 2009
> Running multiple cyrus instances with different dbs ? How do we do that.
> I have seen the ultimate io-contention point is the mailboxes.db file.
> And that has to be single.
> Do you mean dividing the users to different cyrus instances. That is a
> maintenance issue IMHO.
As Bron said, yes it is, but if you have more than 1 machines worth of users
anyway, you have maintenance issues anyway. So rather than just one instance
per machine, we run multiple instances per machine. The only issue it really
introduces is that folder sharing between arbitrary users isn't possible
(unless you used murder to join all the instances together again, but we
don't), only users within an instance can share.
> I had the feeling whatever optimizations done at the FS level would give
> us a max of 5-10% benefit.
> We migrated from ext3 to reiserfs on our cyrus servers with 30k
> mailboxes. I am not sure I saw a great benefit in terms of the iowait.
> At peak times I always see a iowait of 40-60%
To be honest, that's not what we saw in our ext3 <-> reiserfs tests.
What mount options are you using? Are you using the mount options I
> But the new Solid-State-Disks seem very promising. They are claimed to
> give 30x the throughput of a 15k rpm disk. If IO improves by 30 times
> that should make all these optimizations unnecessary.
> As my boss used to tell me ... Good hardware always compensates for
> not-so-good software.
What we've found is that the meta-data (eg mailbox.db, seen db's, quota
files, cyrus.* files) use WAY more IO than the email data, but only use
1/20th the space.
By separating the meta data onto RAID1 10k/15k RPM drives, and the email
data onto RAID5/6 7.2k RPM drives, you can get a good balance of
More information about the Info-cyrus