craig at postnewspapers.com.au
Fri Nov 14 02:22:42 EST 2003
> Certainly one can just copy the entire mail spool... getting MOST of hte
> messages, and the .headers, .cache, .index, etc. Is this safe?
Personally, I wouldn't just copy the spool from beginning to end while
cyrus was running - all your indexes etc will become out of sync if mail
is delivered to a mailbox while it's being copied. *ick*. You also run
the risk of a mangled mailboxes.db if it's being modified while it's
What I'm doing here is using LVM on linux to take a snapshot of the mail
spool (actually all of /var). I make a plain-text copy of the
mailboxes.db then take the snapshot. I mount the snapshot on a temporary
mountpoint, add imap/, spool/imap/, /etc/imapd.conf, /etc/cyrus.conf and
/usr/local/cyrus (where all the binaries etc live on my system) into a
CPIO archive, gzip it, and scp that to another host. That host later
writes the cyrus backup archive to tape.
This is automated and happens nightly. Additionally, both the most
recent and second-most-recent backup files are stored live on the other
server, so I can quickly recover from a failure of the primary cyrus
server that doesn't affect anything else.
Note that my method does /not/ ensure that Cyrus isn't modifying the
mail spools as I take the snapshot, nor does it ensure that mailboxes
aren't changed between the conversion of the mailboxes.db to text and
the snapshot. It's not perfect. For my site, however, it's quite
sufficient - at worst I'll have an AWOL mailbox or two, a mailbox
slightly out of sync with it's indexes, or a truncated message.
I'm also running into an issue where occasionally the snapshot fails -
it's taken while reiserfs is in the middle of a transaction, and the
journal can't be replayed on mount because the snapshot is read-only.
This is more of an LVM issue than anything else. I get around it by
syncing the disk before taking the snapshot, and by deleting the
snapshot and trying again if it's not useable.
I'd love to know of a way to ask cyrus to defer message delivery briefly
and make sure everything is in sync. That way I could signal cyrus to
"sync up and pause", do the db conversions, snapshot the volume, and
resume cyrus. I'd prefer to avoid killing and restarting the master
during the normal backup process.
> What about the databases? Are they save to just copy? are they
> recoverable? Can they be rebuilt from the plain messages?
My understanding is that the mailbox contents can be rebuilt from the
messages, though I seem to remember you'll lose all your flags like
read, etc. The mailboxes.db _can not_ be reconstructed easily and must
be backed up; the rest, I just don't know enough about. I understand
that you can afford to lose the rest of the DBs.
> Isn't that
> what matters most, the messages?
The messages and and your mailboxes DB, yes. Though users will be /very/
unhappy if you lose their subscriptions and/or read status. Recovery
without a copy of your mailboxes db is, I understand, a total
nightmare. I make a plaintext copy so that if I'm restoring the mail
spools etc onto a host with a different berkley DB environment I can get
up and running quickly and easily.
Anyway, I'm extremely far from an expert on this, having been running a
Cyrus server for only a few months. If I'm wrong on some points, please
let me know folks.
This area is one where I'm going to try to add some info to the wiki.
There's some there, but it seems a little sparse and maybe I can help.
More information about the Info-cyrus