Hot backups

[Edward Rudd]

Actually I do have a perl script that I wrote that recreates the
mailboxes.db file if it is lost.. I had this happen when I was upgrading
from 2.0 to 2.1 on my system (my personal mail box only).. 
The script only does the user directory and just adds in all the folders
with the permissions of cyrus all and the user all for all folders into
a text file and runs the ctl_mailbox command to take the plain text
version of mailboxes that the script generates to cyrus's DB.

huh. for some reason I can't find that script..hmm. Well, if anyone
needs it I'll hunt for it.. it's on one of my systems..

On Fri, 2003-11-14 at 01:22, Craig Ringer wrote:
> > Certainly one can just copy the entire mail spool... getting MOST of hte
> > messages, and the .headers, .cache, .index, etc. Is this safe?
> 
> Personally, I wouldn't just copy the spool from beginning to end while 
> cyrus was running - all your indexes etc will become out of sync if mail 
> is delivered to a mailbox while it's being copied. *ick*. You also run 
> the risk of a mangled mailboxes.db if it's being modified while it's 
> being copied.
> 
> What I'm doing here is using LVM on linux to take a snapshot of the mail 
> spool (actually all of /var). I make a plain-text copy of the 
> mailboxes.db then take the snapshot. I mount the snapshot on a temporary 
> mountpoint, add imap/, spool/imap/, /etc/imapd.conf, /etc/cyrus.conf and 
> /usr/local/cyrus (where all the binaries etc live on my system) into a 
> CPIO archive, gzip it, and scp that to another host. That host later 
> writes the cyrus backup archive to tape.
> 
> This is automated and happens nightly. Additionally, both the most 
> recent and second-most-recent backup files are stored live on the other 
> server, so I can quickly recover from a failure of the primary cyrus 
> server that doesn't affect anything else.
> 
> Note that my method does /not/ ensure that Cyrus isn't modifying the 
> mail spools as I take the snapshot, nor does it ensure that mailboxes 
> aren't changed between the conversion of the mailboxes.db to text and 
> the snapshot. It's not perfect. For my site, however, it's quite 
> sufficient - at worst I'll have an AWOL mailbox or two, a mailbox 
> slightly out of sync with it's indexes, or a truncated message.
> 
> I'm also running into an issue where occasionally the snapshot fails - 
> it's taken while reiserfs is in the middle of a transaction, and the 
> journal can't be replayed on mount because the snapshot is read-only. 
> This is more of an LVM issue than anything else. I get around it by 
> syncing the disk before taking the snapshot, and by deleting the 
> snapshot and trying again if it's not useable.
> 
> I'd love to know of a way to ask cyrus to defer message delivery briefly 
> and make sure everything is in sync. That way I could signal cyrus to 
> "sync up and pause", do the db conversions, snapshot the volume, and 
> resume cyrus. I'd prefer to avoid killing and restarting the master 
> during the normal backup process.
> 
> > What about the databases? Are they save to just copy? are they
> > recoverable? Can they be rebuilt from the plain messages?
> 
> My understanding is that the mailbox contents can be rebuilt from the 
> messages, though I seem to remember you'll lose all your flags like 
> read, etc. The mailboxes.db _can not_ be reconstructed easily and must 
> be backed up; the rest, I just don't know enough about. I understand 
> that you can afford to lose the rest of the DBs.
> 
> > Isn't that
> > what matters most, the messages?
> 
> The messages and and your mailboxes DB, yes. Though users will be /very/ 
> unhappy if you lose their subscriptions and/or read status. Recovery 
> without a copy of your mailboxes db  is, I understand, a total 
> nightmare. I make a plaintext copy so that if I'm restoring the mail 
> spools etc onto a host with a different berkley DB environment I can get 
> up and running quickly and easily.
> 
> Anyway, I'm extremely far from an expert on this, having been running a 
> Cyrus server for only a few months. If I'm wrong on some points, please 
> let me know folks.
> 
> This area is one where I'm going to try to add some info to the wiki. 
> There's some there, but it seems a little sparse and maybe I can help.
> 
> Craig Ringer
> 
-- 
Edward Rudd <eddie at omegaware.com>
Home Page <http://urkle.drip.ws/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.andrew.cmu.edu/mailman/private/info-cyrus/attachments/20031114/3710431e/attachment.bin