load balancing at fastmail.fm

[David Carter]

On Mon, 12 Feb 2007, Marten Lehmann wrote:

> what do you think about moving the mailspool to a central SAN storage 
> shared via NFS and having several blades to manage the mmapped files 
> like seen state, quota etc.?

Why do you need NFS?

The whole point of a SAN is distributed access to storage after all :).

> So still only one server is responsible for a certain set of mailboxes, 
> but these SAN boxes have nice backup and redundancy features which are 
> hard to get with common servers

It depends how much you trust your SAN.

Some of my colleagues who run a SAN have had no end of grief. At which 
point you are dependant on the abilities of the vendor to diagnose and fix 
problems. It was this experience that encouraged me to try application 
level replication with lots of small servers in the first place. At least 
that way I can keep a close eye on what the various copies are up to.

A SAN doesn't protect you if your filesystem decides to explode: I believe 
that Fastmail have direct experience of this. Two independent copies of 
the data allows you to keep running a service for the hours that an fsck 
typically takes to complete with file per msg stores on large modern 
disks. It also means rather less stress if the fsck fails to complete. 
I've heard horror stories about all the common Linux filesystems and I've 
personally watched fsck.ext3 (supposedly the safest option) unravel a 
filesystem, with thousands of entries left in lost+found. ZFS looks nice.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.