Cyrus IMAP and MySQL mailboxes (Building load-balancing cluster)

[Marcelo Maraboli]

Andrew

thanks for the "scalable" Cyrus solutions, but I´m wondering
what can be done for Availability solutions ??

What if an IMAP server dies (we had this happen) ??

We have a Solaris Server with a RAID5 disk array storing the
MBOX, but the server died....so downtime was a bit huge..

I wan to build a 100% available IMAP solution...is there any?

regards,

Andrew Morgan wrote:
> On Fri, 17 Nov 2006, Igor Zhbanov wrote:
> 
>> 2006/11/17, Adam Tauno Williams <adam at morrison-ind.com>:
>>> > Yes, I know how failover cluster works. But what if one server
>>> > (active) can't process such a load? Suppose, we plan to have 100 000
>>> > users working actively with mail. I understand that it is possible to
>>> > use one monstrous server to take all of the load, but I am interested
>>> > in load-balancing solution on relatively inexpensive servers.
>>>
>>> (a) SANs are not that expensive.
>>> (b) SANs are *WAY* *WAY* more reliable than *ANY* storage solution you
>>> can build yourself for the same amount of money.  If you really don't
>>> believe that you need to lay of smoking the good stuff.  And (b.1) - if
>>> you have that many users but can't afford a SAN...
>>> (c) Then there is Cyrus replication and there is GFS.  There was long
>>> thread on Cyrus IMAP, HA, & GFS just back in October.
>>>
>>>
>>> > And what
>>> > about slow anti-viruses for 100 000 users' mail? Or to use
>>> > load-balanced front-ends connected to single SAN and connected to
>>> > anti-virus load-balanced cluster? :-)
>>>
>>> It doesn't require a cluster to load balance SMTP,  traditional and well
>>> established technologies will do that for you.  Setup multiple SMTP
>>> servers and publish multiple MX records.
>>
>> Yes, I can use DNS-based load-balancing to spread load to several
>> frontends. But what about backends? :-) How to balance load for them?
> 
> There are basically 3 known solutions to building a scalable Cyrus system:
> 
> 1. Cyrus murder + Cyrus replication.  Cyrus murder distributes mailboxes 
> across multiple backend servers.  The murder backend servers could use 
> local storage, or be connected to a SAN, non-shared file system.  The 
> murder frontends (you can run as many frontends as you want) accept 
> incoming IMAP and LMTP connections and route them to the correct murder 
> backend server.  You could use DNS round-robin to load balance 
> connections between the murder frontends, or you could use something 
> more sophisticated like LVS or a hardware-based network load balancer.  
> Use Cyrus replication to keep a backup copy of each murder backend.
> 
> 2. Cyrus replication + perdition/nginx.  Manually distribute your 
> mailboxes between multiple Cyrus servers (in a non-murder 
> configuration). Use Perdition or nginx to route incoming IMAP 
> connections to the correct server.  Use Cyrus replication to keep a 
> backup copy of each murder backend.
> 
> 3. Cyrus + SAN + clustering.  Use multiple servers in a cluster, 
> connected to a SAN.  Several different people have attempted this 
> according to recent mailing list postings here.  The only successful 
> cluster I'm aware of was a Tru64 cluster.
> 
>     Andy
> ----
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
> 

-- 
Marcelo Maraboli Rosselott
Jefe Area de Redes y Comunicaciones (Network & UNIX Systems Engineer)
Ingeniero Civil Electronico, CISSP       (Electronic Engineer, CISSP)

Direccion Central de Servicios Computacionales (DCSC)
Universidad Tecnica Federico Santa Maria        phone: +56 32 2654071
Chile.                 http://www.usm.cl   http://elqui.dcsc.utfsm.cl