Cyrus IMAP and MySQL mailboxes (Building load-balancing cluster)

[Sarah Walters]

Marcelo et al, 

> -----Original Message-----
> From: info-cyrus-bounces at lists.andrew.cmu.edu 
> [mailto:info-cyrus-bounces at lists.andrew.cmu.edu] On Behalf Of 
> Marcelo Maraboli
> 
> thanks for the input, I know wishing 100% is only available
> with a gooooogle size amount of money ;), but I am looking
> for a CYRUS IMAP server solution similar to a load balancing
> web server farm...i.e:
> 
> - a Load balancing server (PEN in Freebsd if you like) that
> will direct an IMAP session to ANY of a group of IMAP servers,
> all of which have access to a central storage of user MBOXs.
> 
> So if any of the IMAP (backend) server dies, the load balancer with
> automatically not forward any new requests to that server
> and users won´t notice any downtime..
> 
> this is diferent from Andrew´s solution number 1, since ANY of
> the backend IMAP server should accept connections for ANY user.
> 
> examples:
> http://siag.nu/pen/vrrpd-linux.shtml
> http://redundancy.org/fbsd_lb.html
> 
> can IMAP be set up this way ??
> 
> regards,
> 

This need is why I suggested beefy servers rather than the Murder, which I don't consider sufficiently highly available due to actually being a number of discrete servers at the back end. Great for load balancing, useless for instant failover in case of server loss.

In short, as I understand it Cyrus cannot be set up this way. Only a single machine can have write privileges to the mailboxes database at a time. The only way I can see to do this is to use NFSv4 which is supposed to get the locking correct. Then, assuming the database is closed between changes (can a developer please confirm whether it is kept open by master or not?) you should be able to run multiple IMAP servers over the same filesystem stored on a NAS (network-attached storage, as opposed to SAN). That is the only way I can think of to do what you are after. You would need two NAS boxes, ideally in separate buildings, with live mirroring (10 Gb fibre or copper connection between) and a bunch of cheap servers in each building all load-balanced. You should be able to lose a complete data centre and just keep running at 50% capacity as long as your network is properly routed (with redundancy in case of an idiot with a spade cutting through your fibre of course).

It's expensive, but it should work if the database is not held open. If it is, then you need to look at a different email product. Cyrus is a great server, but if you need five 9s reliability then you have to pay for it. You could always look at an appliance - dedicated hardware is often more reliable and at least if it goes down you can scream at the vendor and cover your butt that way.

Regards,
Sarah Walters