Too slow

[Robert Mueller]

>> I turned the mail partition now to xfs and it's terribly fast. WOW!!!
>
> did you use an ext3 partition with dir_index before? I'm just asking

We've just had some experience with filesystems ourselves. Previously we've 
used reiserfs exclusively for the last 5 years. All up, it's been really 
good to us. There are two really, really important things with reiserfs 
though:

1. You MUST have hardware that doesn't lie about it's write cache. When the 
filesystem tells the device driver to sync to disk, and the disk says it's 
done, it must be done (http://community.livejournal.com/lj_dev/670215.html - 
see the Disk cache issues)
2. Your hardware must be IO reliable, it must never report any "write" or 
"read" IO errors at the sector level

Both of these issues are really to do with your hardware and device driver. 
If you've got good hardware and drivers, you should never see either. I 
believe that every report out there with people saying "reiserfs screwed my 
partition totally" is because of one of these 2 things. In the 5 years we've 
been using it, we've had dozens of different kernel crashes, power lost, 
etc, and not one corruption due to any of it. The only corruptions we've had 
are due to IO errors on the external RAID device itself (RAID 6, two 
simultaneous failed drives, and a third drive started reporting errors which 
were returned as IO errors to the kernel). In this respect, IBM seem to 
really have gotten their SCSI hardware right.

Recently, a number of new machines we got we decided to try out ext3 again 
with dir_index. The results have been horrible with huge server loads. Using 
a spare partition, we're juggling users back to reiserfs partitions. The 
result is significantly lower loads for the exact same user set.

I'd rate the general pros/cons of *linux* filesystems as:

* ext3
pros: most widely used; excellent recovery tools; full data journaling 
available; best in the face of flakey hardware or disk caches that lie
cons: performance just isn't that good in a large active user base

* reiserfs
pros: performs well with large active user base configuration, full data 
journaling available
cons: recovery tools generally work, but have been known to crash and can be 
slow on large partitions; large mount time (will be fixed in 2.6.19), 
apparently some concurrecny issues with taking the BKL

* xfs
pros: fast on large files, good concurrency
cons: no data journaling, only meta-data; not really "stable" when bugs like 
this occur that even a xfs_repair wouldn't fix! 
(http://oss.sgi.com/projects/xfs/faq.html#dir2)

All the other filesystems I'd label as less used, which means that it's more 
likely bugs to appear and wouldn't recommend for a production environment.

What about zfs? It's solaris only, and I believe there's still horrible 
performance in the face of fsync() calls, which cyrus does a LOT of.

My 2 cents.

Rob