Cyrus Replication (example) [was Re: restore from cyrdump]
Nic Bernstein
nic at onlight.com
Thu Dec 18 10:59:25 EST 2014
Folks,
Following Willy Offermans' recent response to this thread, I've decided
to jump in, but wish to quote a bit more of Patrick's original message
(below), hence the out-of-sequence reply.
I'd like to address Patrick's issues in pretty much the order they're
presented here. Firstly, however, I would like to echo some confusion
as to the purpose of cyrdump. It seems to have been around since at
least 2.2.3, but documentation has appeared only fleetingly, and says
little about the program:
NAME
cyrdump - dump mailboxes to stdout
SYNOPSIS
cyrdump [-C <configfile>] [-v] [mboxpattern ...]
DESCRIPTION
A tool for dumping IMAP mailboxes on a server.
-C <configfile>
Specify an alternate configuration file ( is used by default)
-v Increase program verbosity.
Not much help, that.
As for the utility of filesystem snapshots, be they LVM2, ZFS, XFS,
etc., this of course extends only so far as the stability of the
underlying data. If your data is in flux at the time of the snapshot
(unwritten buffers, etc.) then you'll only snapshot an incoherent, or
"crash-consistent", state. Thus it's important to consider the relative
cost and consequence of taking the extra steps to quiesce the
application prior to the snapshot.
We perform ZFS snapshots very frequently, without application
quiescence, under the belief that even crash-consistent data is better
than no data, in a DR scenario, and we replicate this data across
multiple machines and sites, to further improve any potential DR
outcome. However, we also make "application-consistent" backups, which
require application quiescence, as Patrick has described. His initial
suggestion -- stop cyrus, snapshot, restart cyrus -- is reasonable, but
we feel that the later suggestion -- stop cyrus, tar up data, start
cyrus -- is not. It takes data offline for too long. That's why the
snapshot capability is necessary in any truly suitable server.
If you, like us, maintain your "config" and spool (and potentially meta)
data on separate filesystems, you must remember to snapshot them all.
Also, if you're going to stop and start cyrus like this, it pays to
remove heavyweight processes (i.e. delprune, checkpoint, sqautter) from
the START section of cyrus.conf, to prevent extensive delays at restart,
and let them be performed periodically (in EVENTS) instead.
I will say that the ability to quiesce the application without halting
it would be most desirable. Most databases have supported this sort of
thing for ages, and it would be great if one could send a signal to
Cyrus to achieve the same result. One wonders, however, if the model of
master process (cyrmaster for Debian-based distros) with potentially
hundreds or thousands of child daemons, would lend itself to this.
But snapshots, whatever their consistency or coherence, will always be
less useful for DR than a replica, and in Cyrus this is not only
available, but not really that hard to get up and running. The
differences between the imapd.conf files on a master and replica can be
this simple, or simpler even, as this example is from a murder (from a
running pair):
--- imapd-master.conf 2014-11-24 13:37:45.752639666 -0600
+++ imapd-replica.conf 2014-11-24 13:31:58.702611566 -0600
@@ -1,22 +1,14 @@
##
-# These configuration parameters are for the master server
-# in a replication set
-servername: master.example.com
+# These configuration parameters are for the replica server in a
+# replication cluster
+servername: replica.example.com
-##
-# Auth credentials
-mupdate_server: mupdate.example.com
-mupdate_username: mupdate
-mupdate_authname: mupdate
-mupdate_password: <password>
-
-##
-# Replication support
-# This is how the BACKEND for this host is defined
-sync_host: replica.example.com
-sync_authname: mailproxy
-sync_password: <password>
-sync_compress: true
-sync_log: true
-sync_repeat_interval: 5
-sync_shutdown_file: /var/run/cyrus/sync_stop
+## Auth credentials
+# The credentials below must match the account listed in lmtp_admins
+# on the backend servers.
+proxy_authname: mailproxy
+proxy_password: <password>
+force_sasl_client_mech: PLAIN
+mupdate_mechs: PLAIN
+mailbox_mechs: PLAIN
+serverlist: master
The changes in cyrus.conf are similarly brief:
--- master/cyrus.conf 2014-11-24 14:01:26.832602819 -0600
+++ replica/cyrus.conf 2014-11-24 14:01:42.212637226 -0600
@@ -15,8 +15,8 @@
# Master sends mailbox updates to mupdate.
# Replication client runs on Master.
# Comment these 2 lines out on replicas
- #mupdatepush cmd="/usr/sbin/cyrus ctl_mboxlist -m"
- #syncclient cmd="/usr/lib/cyrus/bin/sync_client -r"
+ mupdatepush cmd="/usr/sbin/cyrus ctl_mboxlist -m"
+ syncclient cmd="/usr/lib/cyrus/bin/sync_client -r"
# this is recommended if using duplicate delivery suppression
#delprune cmd="/usr/sbin/cyrus expire -E 3 -D 60 -X 60" at=0400
@@ -55,7 +55,7 @@
##
# Synchronization for remote replication.
# Comment this out on Master, uncomment on replicas
- syncserver cmd="/usr/lib/cyrus/bin/sync_server" listen="csync"
+ #syncserver cmd="/usr/lib/cyrus/bin/sync_server" listen="csync"
# Experimental httpd for caldav
httpd cmd="httpd" listen=8080 prefork=1 maxchild=20
It's just not that hard to implement, and for the security and safety it
provides is well worth it.
Note that if you're using a murder, the server name must be preserved
between the two hosts, in the event of a fail-over, and only the master
should communicate with the mupdate server (as shown in my example). If
you're not using a murder, then most of these differences go away (all
the mupdate stuff). Of course, you'll need to switch DNS or have some
other way of dealing with fail-over on layer 3.
We put all common configuration into /etc/imapd.conf, and then use the
@include directive to include the appropriate file fragment, either
imapd-master.conf or imapd-replica.conf. For the cyrus.conf, it's just
a couple of line edits -- commenting and uncommenting -- so we do that
by hand. You've got to intervene to update the DNS, in any event, so
this much more is trivial.
Lastly, as to the use of imapsync to achieve user, mailbox or server
replication, and the authentication requirements thereof, I would
suggest reading the README file for that application, which includes this:
You may authenticate as one user (typically an admin user), but be
authorized as someone else, which means you don't need to know every
user's personal password. Specify --authuser1 "adminuser" to enable this
on host1.
So your command line is much like Patrick's example, but with '--user1
<user> --authuser1 <proxyuser> --user2 <user>...'
Of course you must create a proxy user, and Cyrus supports this with the
'proxyserver' directive in imapd.conf (man imapd.conf for details),
i.e.: 'proxyservers: proxyuser'.
However, I must be honest and point out that if you're going to go to
the trouble of figuring out how to use imapsync (and possibly pay for
it, to boot) you may as well just set up a replica. As I've shown,
above, it's just not that hard.
Cheers,
-nic
On 12/15/2014 05:01 PM, Patrick Goetz wrote:
> This still leaves the question of how best to back up a cyrus mailstore.
>
> Bron mentioned that most people are using LVM snapshots. I don't see
> how using btrfs/LVM/ZFS snapshots can save you from a race condition
> between when the cyrus user directory is updated and when mailboxes.db
> is updated. The only way I would trust this is by doing this:
>
> 1. Stop cyrus
> 2. Snapshot
> 3. Restart cyrus
>
>
> cyrdump: near as I can tell the only useful purpose this serves is to
> assemble all email messages into a single "mbox" file (can anyone
> confirm this)?
>
> ctl_mboxlist: this seems useful for making a human readable copy of the
> mailboxes.db file, but I'm not sure how this could be useful for
> disaster recovery, given the previously mentioned issue about keeping
> the mailboxes.db file synchronized with the contents of the user dir.
>
> So, given a simple mail server (i.e. no murder + replication), and when
> using a filesystem (e.g. ext4 or XFS) which doesn't do snapshots, it
> would appear that the only safe way to backup up a cyrus mailstore is to
> either using something like imapsync, or
>
> 1. Stop cyrus
> 2. tar cvf /some/safe/place/user.tar {default-partion}
> 3. tar cvf /some/safe/place/cyrusdb.tar {configdirectory}
> 4. Restart cyrus
>
> The way I've used imapsync in the past required copying mail folders per
> authenticated user account; i.e. something like
>
> imapsync --host1 my_host1 --authmech1 LOGIN --user1 my_user1 --password1
> xxxxx --host2 my_host2 --authmech2 PLAIN --user2 my_user2 --password2 xxxxx
>
> which in particular means knowing everyone's passwords. This is
> entirely unworkable for larger sites, and I'm not sure if there is a
> trick for getting around this.
--
Nic Bernstein nic at onlight.com
Onlight, Inc. www.onlight.com
219 N. Milwaukee St., Suite 2a v. 414.272.4477
Milwaukee, Wisconsin 53202
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20141218/8560752b/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nic.vcf
Type: text/x-vcard
Size: 271 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20141218/8560752b/attachment-0001.vcf
More information about the Info-cyrus
mailing list