Backup strategy for large mailbox stores

[Clement Hermann (nodens)]

On 16/02/2010 19:45, Vincent Fox wrote:
> Andrew Morgan wrote:
>> Is there really a significant downside to performing backups on a hot
>> cyrus mailstore?  Should I care if Suzie's INBOX was backed up at 3am
>> and Sally's INBOX was backed up at 4am?
>>
>> Vincent, on a slightly related note, what is your server and SAN
>> hardware?
>>
> I dunno, perhaps the Cyrus gurus could answer that better.
> I rather assumed I would want my meta information to match
> fairly closely the contents of the inboxes at that point in time.
> My belief was that if I had to a full restore after a disaster
> that I would have to spend substantial time doing reconstruct
> in order to get the databases to represent actual state.
> Thus having a point-in-time snapshot would be better for DR.
> Perhaps I'm wrong about that.

Our main (and I suppose, most people's) concern with hot backup on a 
busy server, is that the backup has significant impact on disk 
performance (we use integrated raid 10 card with 10krpm 2,5" SAS disks, 
no san). Also, it does take a lot of time, so if the backup start after 
the last I/O peak (say, 22:00), there are chances that it won't be 
finished at 8:00 when people start using the server heavily again.

The snapshot approach (we use ext3 and lvm, soon ext4) is promising, as 
a simple tar is faster than using the full backup suite on a filesystem 
with a lot of small files (atempo here). But you need the spare space 
locally, or you need to do it over the network, and it will take time 
(but won't probably kill disk I/O as much as the backup software).

Bron's solution at fastmail seems pretty elegant to me, but may be a bit 
hard to implement. Writing that much custom scripts is not very 
appealing right now, as we don't have many spare time currently, but it 
is definitely something I will look into.

The cold standby server for redundancy using imap replication, where you 
backup only the standby server that is light on I/O, is also 
interesting, but we don't use 2.3 yet (we plan to), but then it would be 
best to have two mail stores on each servers, on different I/O cards, 
one of them being the primary store for half the users, the other one 
being the replicate (and the one that is backed-up).


Cheers,

-- 
Clement Hermann (nodens)
- "L'air pur ? c'est pas en RL, ça ? c'est pas hors charte ?"
Jean in L'Histoire des Pingouins, http://tnemeth.free.fr/fmbl/linuxsf/

Vous trouverez ma clef publique sur le serveur public pgp.mit.edu.
Please find my public key on the public keyserver pgp.mit.edu.