Backup methods

[Anatoli]

 > Not very sure to understand that. It's always true isn't ? If you 
have XTo of data and you want n backups you will need X*(n+1) To ?

The replication as it is designed means that you create an additional 
(replica) instance of Cyrus that will be in sync with the master 
instance, so when you need to make a backup, you turn of the replica, 
take a backup from /its data/, then turn it on again so it comes in sync 
with the master. In this case there's no interruption to the service, 
you just stop a replica. But the replica will use the same amount of 
space as your master, so without even making a backup, you'll use 2x 
space. + you have to understand how the replication works, then set it 
up, control that the sync process is always working and the replica has 
the same information as the master... That's a great solution for 
ISP-level or public mail service operations, but IMO an absolute 
overkill for small deployments.

 > I don't see how you can avoid that, of course you can activate heavy 
compression on the backup but beside of that....

When it comes to making a backup, the best policy IMO is to make 
incremental backups. In this case you only store the new mails + binary 
indexes. Once in a while (e.g. every month) you make a full backup, 
then, say, once a week a level 1 backup (that stores changes from the 
previous week, reset at lower level backup, i.e. every month), then 
daily level 2 backups and hourly level 3. This way you can restore up to 
hourly changes without using excessive amount of space. Of course you 
can compress them too (xz -9 gives a pretty good ratio).

 > Well that's is easy to avoid, you just have to stop postfix before 
stopping the VM, when postfix is stop all incoming messages will stay on 
the parent smtp server, so no loosing incoming mail.

Uhh don't do that. Your Postfix has no problem in retaining mails if 
Cyrus is not reachable, then attempt their delivery again. I was 
referring to that, depending on the configuration of your incoming MTA, 
the next delivery attempt may be in, say, 15 minutes, so you postpone 
incoming mail for that time if you turn off Cyrus to take a backup. If 
you turn off your /incoming MTA/, the source MTA may have issues with 
delivery at all (you don't control it, you don't know how it's 
configured, when the next delivery attempt will occur, etc.), never turn 
off your incoming MTA.

*From:* Albert Shih
*Sent:* Thursday, May 10, 2018 04:14
*To:* Anatoli
*Cc:* Info-cyrus
*Subject:* Re: Backup methods

Le 10/05/2018 à 02:44:18-0300, Anatoli a écrit

Hi,

> The replication is reasonable only if you have more than one server in your
> deployment (and both servers with the same level of security, if not you risk
> to compromise the user data) or "spool size/available disk space" is low,
> otherwise you'd need to dedicate 2 times more space than needed to store user
> data, only to take a periodic backup (+ the space needed to store the backup
> itself).

Not very sure to understand that. It's always true isn't ? If you have XTo
of data and you want n backups you will need X*(n+1) To ?

I don't see how you can avoid that, of course you can activate heavy
compression on the backup but beside of that....

> I suggest you take a look at this issue: https://github.com/cyrusimap/
> cyrus-imapd/issues/1763, where backups for small deployments were already

Thanks for the link I will read that.

> Answering the OP's question, I'm using Cyrus for 4 years now and I don't know
> about any reliable and reasonable strategy for backups of Cyrus data in SME
> environments. Summing it up:
>
>    • FS snapshots without stopping the server: a possibility of a corrupted
>      backup.
>    • FS snapshots after stopping the server: service downtime, breaking open
>      connections, delivery issues for incoming MTAs, etc. - reasonable for daily

Well that's is easy to avoid, you just have to stop postfix before stopping
the VM, when postfix is stop all incoming messages will stay on the parent
smtp server, so no loosing incoming mail.

>      backups in a 8/5 office, unreasonable for 24/7 deployments (e.g. users
>      distributed in different time zones) or for intra-day backups.

I check, stopping postfix, stopping the VM, take a snapshot, starting the
VM, take about 10-15 secondes. So I agree with you it's not a very good
solution because user still can loose the connection, but I think without
replication it's acceptable.

>    • Replication: unreasonable requirements for disk space, setup overkill.

For the setup the overkill is for me a small price vs loosing data....and
as for the disk space that's not a issue at all for me. Currently I run
dovecot and have 2 backups, so when I say to my boss « we got X To of mail »
I already got 3 * X To of disk. Say in other way, if I can afford X To, I
will say I can give you X/3 To of mail.

Regards.

--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
xmpp: jas at obspm.fr
Heure local/Local time:
Thu May 10 09:03:31 CEST 2018



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20180510/3cc363a9/attachment-0001.html>