restore from cyrdump

[Patrick Goetz]

On 12/10/2014 03:47 AM, Willy Offermans wrote:
> I'm not sure what you mean with ``all the metadata'', but there are user
> flags saved into the cyrdump file as well. I never performed the whole
> cycle of dump and restore (probably nobody did so far), so I cannot tell
> you that all metadata is available in the dump file. See my question above!
>


A while back I was working on an email server that (unbeknownst to me) 
was connected to a UPS, but with an external disk array that was plugged 
in to an outlet on the UPS that was not battery-backed.  This site had 
frequent power problems, so it turns out that power cycling a disk array 
while the server stays up is an awesome way to corrupt your entire file 
system.

Since I didn't know what I was doing at the time, I restored

   partition-default: /home/cyrus

without also restoring

   configdirectory: /var/lib/cyrus

I was consequently confused when no mailboxes showed up, and had to then 
learn about and use reconstruct -r on each individual mailbox 
(cyrreconstruct on debian/Ubuntu) in order to reconstruct the 
/var/lib/cyrus/*.db files.

I think the main database files you need are mailboxes.db and 
annotations.db (can someone confirm this?)

This still leaves the question of how best to back up a cyrus mailstore.

Bron mentioned that most people are using LVM snapshots.  I don't see 
how using btrfs/LVM/ZFS snapshots can save you from a race condition 
between when the cyrus user directory is updated and when mailboxes.db 
is updated.  The only way I would trust this is by doing this:

    1. Stop cyrus
    2. Snapshot
    3. Restart cyrus


cyrdump:  near as I can tell the only useful purpose this serves is to 
assemble all email messages into a single "mbox" file (can anyone 
confirm this)?

ctl_mboxlist: this seems useful for making a human readable copy of the 
mailboxes.db file, but I'm not sure how this could be useful for 
disaster recovery, given the previously mentioned issue about keeping 
the mailboxes.db file synchronized with the contents of the user dir.

So, given a simple mail server (i.e. no murder + replication), and when 
using a filesystem (e.g. ext4 or XFS) which doesn't do snapshots, it 
would appear that the only safe way to backup up a cyrus mailstore is to 
either using something like imapsync, or

    1. Stop cyrus
    2. tar cvf /some/safe/place/user.tar {default-partion}
    3. tar cvf /some/safe/place/cyrusdb.tar {configdirectory}
    4. Restart cyrus

The way I've used imapsync in the past required copying mail folders per 
authenticated user account; i.e. something like

imapsync --host1 my_host1 --authmech1 LOGIN --user1 my_user1 --password1 
xxxxx --host2 my_host2 --authmech2 PLAIN --user2 my_user2 --password2 xxxxx

which in particular means knowing everyone's passwords.  This is 
entirely unworkable for larger sites, and I'm not sure if there is a 
trick for getting around this.