best filesystem for imap server

[Henrique de Moraes Holschuh]

On Wed, 01 Dec 2004, Jim Miller wrote:

> > notice is that when the mail delivery queue on the MTA gets very large,
> > which happens occassionally, the CPU load average goes way up and iowait
> > time as displayed using top can exceed 300% on a four processor box and
> > performance

Are you keeping tabs on the number of LMTP processes, and on locking
contention?  You can teach a new-enough postfix to be *very* intelligent on
what the heck it is doing when doing LMTP queueing and delivery.

> > goes all to heck.  Is switching the filesystem to XFS likely to help this
> > situation?  Since there is some 80GB of mail spool currently in use,
> > switching the filesystem to XFS is not a simple task and I don't
> > won't to do
> > it on a lark.

AFAIK the best you can go for the MTA (and probably Cyrus, but I have not
tested) using non-commercial solutions is to get a few gigabytes worth of
fast nonvolatile RAM drives, and use ext3 in data=journaled mode and an
external journal (that goes in the nonvolatile RAM drive), and with all
hashing enabled. This requires a 2.6 kernel.  This is far safer than XFS,
which *will* corrupt queue and message data on crash.  

I tfind it kinda patetic that we cannot tell XFS (in Linux. I don't know
about SGI) to sync for real when fsync() is called, especially since write
barriers ARE fully implemented in the SCSI path for Linux AFAIK.  Maybe it
is just a bug, but until this gets fixed...

However, you can probably fix things (at the expense of far higher latencies
on mail delivery when the queue is huge, and a deeper queue too) by doing a
good job of limiting the LMTP resource usage.  At least the system will be
responsive, even if it will take a while for new mail to get through.

> I feel that XFS is a bad choice since it is not a 'truly' journaled file
> system.  If you have a power failure/system crash/lockup, etc., etc. You
> could very easily end up with a corrupt file system -- XFS doesn't write out
> to the disks immediately (caching unwritten data to memory).  EXT3 is
> journaled but very slow.  ReiserFS is a better choice for a journaled file
> system and if you can hold off until all the bugs are worked out, Reiser4FS
> would be the best choice (IMHO).

I didn't know reiser 3 would fully journal data (or that it has good enough
write barriers and write optimization to make sure the filesystem never
returns before a fsync really means everything including data is on disk).
Is that correct?  If it is, then reiser might be a better choice than ext3
with hashing (as long as you do use a fast-as-heck nvram drive for the
journal, of course).

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html