Backup methods

[Anatoli]

Hi,

> On larger systems with VMs i take a ZFS or LVM snapshot and mount it
> externally to "fetch" a full (incremental) filesystem backup of the mail spool
> and imap spool and cyrus db on a daily base. After the backup run i destroy
> the snapshot.

The problem with this is that you can't be sure the data on disk is in sync. Depending on how heavy the load is during the "backup" (+ your luck), you may find unpleasant surprises when you have to restore it. And you'd only know that the backup is corrupted when you try to restore it.


> Beside this and depending from your needs you may take a look at cyrus
> replication features to build a "backup" or just use standard filesystem
> backup tools like tar, dumpfs etc.

The replication is reasonable only if you have more than one server in 
your deployment (and both servers with the same level of security, if 
not you risk to compromise the user data) or "spool size/available disk 
space" is low, otherwise you'd need to dedicate 2 times more space than 
needed to store user data, only to take a periodic backup (+ the space 
needed to store the backup itself).

I suggest you take a look at this issue: 
https://github.com/cyrusimap/cyrus-imapd/issues/1763, where backups for 
small deployments were already discussed in detail. Though, no idea if 
there are plans to implement it.

Answering the OP's question, I'm using Cyrus for 4 years now and I don't 
know about any reliable and reasonable strategy for backups of Cyrus 
data in SME environments. Summing it up:

  * FS snapshots without stopping the server: a possibility of a
    corrupted backup.
  * FS snapshots after stopping the server: service downtime, breaking
    open connections, delivery issues for incoming MTAs, etc. -
    reasonable for daily backups in a 8/5 office, unreasonable for 24/7
    deployments (e.g. users distributed in different time zones) or for
    intra-day backups.
  * Replication: unreasonable requirements for disk space, setup overkill.


Regards,
Anatoli

*From:* Niels Dettenbach Via Info-cyrus
*Sent:* Wednesday, May 09, 2018 06:42
*To:* Info-cyrus
*Subject:* Re: Backup methods

Am Mittwoch, 9. Mai 2018, 11:19:54 CEST schrieb Albert Shih:

> I would like to know what's kind of backup method are recommended for
> cyrus-imapd.
>
> My cyrus-imapd host (only one currently) are running under FreeBSD jail
> (something like systemd-nspawn, lxc) & ZFS so I'm intend to use this
> method :
>
>    stop the vm
>    take a zfs snapshot
>    start the vm
>
> send the zfs snapshot on a backup server.

This is relatively inefficient, but a working option if anything from cyrus
data is on that VM - i.e. the complete mail spool and the database files
(possibly plus sieve files). We do similiar on relatively small systems or to
get "intraday backups" only.

On larger systems with VMs i take a ZFS or LVM snapshot and mount it
externally to "fetch" a full (incremental) filesystem backup of the mail spool
and imap spool and cyrus db on a daily base. After the backup run i destroy
the snapshot.

Beside this and depending from your needs you may take a look at cyrus
replication features to build a "backup" or just use standard filesystem
backup tools like tar, dumpfs etc.

On a file base you have to backup the mail spool and the cyrus database files.
If you use SIEVE, backup the SIEVE file pool too. You can restore by just
replacing the files and start cyrus. To get the common database files
"interoperable" it may makes sense to dump then into a machine independent
format for backup if they are in a machine dependent format.

If your restore such a filesystem based backup to a new system which has other
hardware / arch specs or newer / incompatible DB subsystem (instead of
skiplist) you may have to "recreate" indizes and database data. reconstruct -
f may be your friend to "clean" up the transfer / restore.

There are several strategies for backup cyrus - this are just a few.


hth a bit.
good luck,


Niels.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20180510/159939f4/attachment.html>