Recomendations for a 15000 Cyrus Mailboxes

[Wesley Craig]

On 10 Apr 2007, at 12:40, John Madden wrote:
> Indeed, I've seen reiserfs do some very long fsck's, but I've seen the
> same out of ext3.  But for a filesystem of 35 million mail files, I
> figure it's got to beat ext3 on performance, at least.  ...But there
> don't seem to be any stats at this scale to support that.

I've repeatedly seen RAID failures, etc, cause reiserfs to be  
meaningfully unrecoverable.  I've never seen anything like that on  
ext3, but it's only been a couple of years.  At the time we moved off  
reiserfs, ext3 also appeared to be better supported.  We routinely  
reevaluate filesystem choice when we purchase new hardware, typically  
every two years.  Since we're in the process of spec'ing new  
hardware, we'll be reviewing the filesystem choice soon.

As far as relative performance is concerned, as I recall properly  
tuned reiserfs performs slightly better than properly tuned ext3.   
Moving mailbox.db, seen DBs, etc, to their own IO chain does more for  
performance than anything else.  I haven't experimented with running  
with three independent IO channel, but I gather that putting  
cyrus.cache and cyrus.index on their own partitions improves  
performance significantly as well.

Regarding scale, the backend I'm looking at just now has 9.8K user,  
2.3TB of user data in 44M files.  This is one of 23 backends in this  
murder.  Generally speaking, I think replication is the way to go in  
either case.

:wes