fsync() takes about 0.06 second ---makes Cyrus deliver slow

[Phil Brutsche]

Daniel Nilsson wrote:
> Su Li wrote:
> 
>> Daniel,
>>
>> My Linux is red hat 7.2-1 with kernel 2.4.7-10. The file system is ext3.
>> Hard disk it RAID5.

[...]

> The interesting thing here is that you are using ext3. I haven't tried 
> ext3 myself but from what I understand using ext3 you get journaling of 
> both data and metadata where for example reiserfs only journals 
> metadata. This obviously has advantages if you are worried about crashes 
> and such, but at the cost of performance.
> 
> I would try using a different filesystem, you can try both ext2 and 
> reiserfs to see what the performance impacts are and then make a 
> tradeoff on which one to use. Do you fell like you need a journaling 
> filesystem for this task ?

I'm successfully running Cyrus under Linux (Debian 3.0, Cyrus 2.1.5 & 
2.1.9, kernels 2.4.17 & 2.4.19), with my Cyrus partitions on RAID5 - I'm 
using 3ware ATA-RAID cards with Seagate Barracuda IV hard drives.

Some things I see:

1) Su Li, you didn't mention if you're using hardware or software RAID5. 
  If you're using software RAID5, switching to a hardware-based solution 
will increase your performance dramatically.

2) Using ext3 (or any other journalling file system) on a RAID5 is 
suicide in terms of performance, due to having the journal on a volume 
that is already slow on writes.  ext2 *will* be faster.  A good way to 
alleviate the performance bottleneck is to put the journal on a separate 
set of disks.  My servers are set up so that there is a mirror for the 
OS, swap, and journal partitions, and a RAID5 for data - a total of 6 
drives across 2 3ware ATA-RAID cards.

Oh, and a journalling file system is *always* worth it.  It's no fun 
waiting around for a fsck on a 180GB file system :)

-- 

Phil Brutsche
phil at optimumdata.com