synchronization/backup mx

Tue May 15 09:50:26 EDT 2007

Hi Phillipe!

Your question is very valid in the first place; a lot of people are looking for that scenario. (Including one of the sites which I work on right now.)

I think you will have to become clear of some things:

1. An MX is a Mail eXchanger, i.e. Exim, Postfix or the like. The MX does not care about mailboxes, it does not store mails (except when queueing) but it's just a kind of switch for email messages.

2. Cyrus IMAPd is a mailbox server. This is where messages get stored.

So ...

> there is some other thing i want to know. is the backup mx (configured in
> dns) automtically used if my primary mailserver goes down because of
> hardware failure or anything else?

Let's say: It should. This what the RFC says and this is how most MX servers behave. So if you have configured server1 and server2 as MX records for yourdomain.com and someone else out there on the internet wants to send a message to someone at yourdomain.com, the sending MX will do a lookup of MX records for yourdomain.com and most likely try to connect to server1 first and in case it cannot reach it, connect to server2 and try there. In case it cannot reach either one, it will use its retry policy to decide when to try where again. You don't have any influence on what sending MXs of other people do.

So if they are configured faulty, they might never try the 2nd MX server or they might always start with the last one first, whatever.

But I think this is not really what you wanted to know.

If I read your question between the lines, that what you're asking is:

If your MUA (Mail User Agent, Thunderbird, OE, webmailer, ...) wants to access your mailbox in order for you to read mail which you received, how can you implement redundancy there to have a failover between two (Cyrus) IMAP servers. Is that what you're asking?

If that's what you're asking:

When your MUA searches for the IMAP server, there is no indirection with MX records, but you have to enter the DNS name of your IMAP server (such as imap.yourdomain.com) and you can enter just one.

To implement failover redundancy (making sure your users can access their mailbox if one mailbox server goes down) you need to solve two problems:

1. Make sure that you keep two copies of each mailbox in sync on two different IMAP servers.
2. Make sure that if one of them fails, the MUA will connect to the 2nd one.

No, murder is not going to help you here, with neither problem. 

Cyrus murder is like a traffic guidance system which refers you to the physical server where the mailbox is. Imagine a large organisation which has to provide 10000 mailboxes with 5 GB storage each. It will be hard to implement that one one physical server, so you might implement 10 server with 1000 mailboxes on them each. But to avoid having to use IMAP parameters on the clients depending on where the mailbox is located, you can use a murder frontend which will take requests from all clients and refer them to the respective server. This also allows you to move mailboxes between servers transparently, for example if you need to split a server which is getting too full. Without murder you would have to go to the users and tell them to change their IMAP server parameter in their MUA if you move their mailbox.

Note that AFAIK murder is *not* an IMAP proxy, so it does not sit in between the client and the mailbox server, but it just redirects the client to the appropriate server. (The gurus please correct me here in case I am wrong.)

What I don't know is this:

If I have multiple A records for imap.mydomain.com in my DNS and my mail client cannot connect to the first one, will it try the next? This definitely depends on the implementation of the mail client, but does anyone have any experience here? For webbrowsers this is said to be common behaviour and this is how to make sites robust. (Do a dig on google.com for example.)

Now as failures are typically not planned, any kind of scheduled backup is not going to do any good for failover. It might for recovery, but this is not what you want, I understand. Though it's better than nothing. ;-)

For failover (=contingency) you need to keep two independent copies of the mailbox in sync on two different servers. This is where my practicial knowledge ends as well right now, so maybe someone else has answers, but:

I doubt this can be achieved using file system mirroring technology. Even if you mount the mailbox directory from some kind of mirrored cluster filesystem, this still does not mean it would work that two instances of Cyrus IMAPd would manipulate that filesystem structure.

So either there would be some application level mechanism inside Cyrus IMAPd (and I think I have seen there is something like that, but I would have to check again) or your plan B would be:

- provide a fault tolerant file system
- mount it on one server running Cyrus IMAPd
- have a 2nd server as cold-standy
- have a murder frontend that users connect to

Now in case of a failure of the primary server, boot the 2nd server and tell murder that the mailboxes are now on the 2nd server.

Your murder would still be a single point of failure though ...

Ok, as I said, maybe the Cyrus gurus can fill us in.

Does anyone have a redundant IMAP config up and running where failure of a mailbox server would not mean any service disruption? It might be worth documenting such a setup in the Cyrus Wiki.

Regards,
Torsten

-------- Original-Nachricht --------
Datum: Tue, 15 May 2007 15:04:35 +0200
Von: "Philippe Trolliet" <p.trolliet at novatec-gmbh.de>
An: "Info-cyrus" <Info-cyrus at lists.andrew.cmu.edu>
Betreff: synchronization/backup mx

> hi,
> i´m looking for a strategy to synchronize the primary mailserver to a
> backup/fallback server. i´ve read an article about backup strategy in the
> cyrus wiki. but is there any other way to do this? e.g. with an internal
> component of cyrus?
> scenario:
> a mail is stored in inbox on primary server.
> the server stores a backup of the mail and config over a second network
> interface on the backup/fallback mx.
> is this possible or has this to be done manually with rsync/lvm snapshot?
> can i realize this in some way with murder?
> 
> there is some other thing i want to know. is the backup mx (configured in
> dns) automtically used if my primary mailserver goes down because of
> hardware failure or anything else?
> 
> regards
> philippe
> 
> 
> --
> I am using the free version of SPAMfighter for private users.
> It has removed 6214 spam emails to date.
> Paying users do not have this message in their emails.
> Get the free SPAMfighter here: http://www.spamfighter.com/len
> 
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html