Cyrus IMAP and MySQL mailboxes (Building load-balancing cluster)

[Andrew Morgan]

On Fri, 17 Nov 2006, Igor Zhbanov wrote:

> 2006/11/17, Adam Tauno Williams <adam at morrison-ind.com>:
>> > Yes, I know how failover cluster works. But what if one server
>> > (active) can't process such a load? Suppose, we plan to have 100 000
>> > users working actively with mail. I understand that it is possible to
>> > use one monstrous server to take all of the load, but I am interested
>> > in load-balancing solution on relatively inexpensive servers.
>> 
>> (a) SANs are not that expensive.
>> (b) SANs are *WAY* *WAY* more reliable than *ANY* storage solution you
>> can build yourself for the same amount of money.  If you really don't
>> believe that you need to lay of smoking the good stuff.  And (b.1) - if
>> you have that many users but can't afford a SAN...
>> (c) Then there is Cyrus replication and there is GFS.  There was long
>> thread on Cyrus IMAP, HA, & GFS just back in October.
>> 
>> 
>> > And what
>> > about slow anti-viruses for 100 000 users' mail? Or to use
>> > load-balanced front-ends connected to single SAN and connected to
>> > anti-virus load-balanced cluster? :-)
>> 
>> It doesn't require a cluster to load balance SMTP,  traditional and well
>> established technologies will do that for you.  Setup multiple SMTP
>> servers and publish multiple MX records.
>
> Yes, I can use DNS-based load-balancing to spread load to several
> frontends. But what about backends? :-) How to balance load for them?

There are basically 3 known solutions to building a scalable Cyrus system:

1. Cyrus murder + Cyrus replication.  Cyrus murder distributes mailboxes 
across multiple backend servers.  The murder backend servers could use 
local storage, or be connected to a SAN, non-shared file system.  The 
murder frontends (you can run as many frontends as you want) accept 
incoming IMAP and LMTP connections and route them to the correct murder 
backend server.  You could use DNS round-robin to load balance connections 
between the murder frontends, or you could use something more 
sophisticated like LVS or a hardware-based network load balancer.  Use 
Cyrus replication to keep a backup copy of each murder backend.

2. Cyrus replication + perdition/nginx.  Manually distribute your 
mailboxes between multiple Cyrus servers (in a non-murder configuration). 
Use Perdition or nginx to route incoming IMAP connections to the correct 
server.  Use Cyrus replication to keep a backup copy of each murder 
backend.

3. Cyrus + SAN + clustering.  Use multiple servers in a cluster, connected 
to a SAN.  Several different people have attempted this according to 
recent mailing list postings here.  The only successful cluster I'm aware 
of was a Tru64 cluster.

 	Andy