Robust Cyrus design

[Paul M Fleming]

I still need to finish my documentation with details etc.. but here is a
diagram of our setup. The Linux LVS will be in place this summer --
currently we use round-robin DNS. 

http://www.siumed.edu/~pfleming/development/email/mailsystem.jpg

We use 4 dedicated IMAP backends.

2 frontends serve imap (current custom IMAP proxy -- moving toward
Murder) , Internal SMTP Delivery & internal LDAP queries. 

1 external frontend serves the same services outside our firewall.

4 external (outside firewall) hosts handle internet inbound SMTP traffic

WebMail (www.horde.org) is handled but yet another machine.

Authentication (not shown in diagram) is handled by 3 geographically
disperse dedicated MIT Kerberos KDCs

All SMTP handlers run mimedefang+mcafee+spamassassin. Short circuit
delivery is performed by LDAP routing in Sendmail. Email not resolvable
via LDAP is forwarded to a canonical host for alias & MailMan expansion. 

I agree with Rob more pieces - more parts to break, but the advantage to
our setup is single failures don't tear down the entire system (the most
critical elements are the backends because each stores unique data) Once
we have the LVS in place we'll be able to service portions of the system
without informing users or only effecting portions of our user base in
the case of backends. Additionally, distributing the load out greatly
increases scalability.. This system has grown from 2 backends (also
performing frontend duties) and 1 SMTP handler. 

We currently do 600 concurrent IMAP logins (across the cluster),
reaching 2-3 logins/second per machine. Inbound SMTP from the Internet
we can handle upwards of 3-5 messages per second. We have 2-3k active
IMAP accounts. Under stress testing we were able to reach 500 concurrent
logins per backend with 10 msgs/second inbound SMTP delivery. Under
regular load (not Mydoom, Sobig, etc) we do 1 second delivery times
inbound including Virus Scanning and SpamAssassin checking. 

Hope this helps. We have EXCELLENT stability using Cyrus and users are
happy with the performance.. 

Paul Fleming
SIU School of Medicine

Happy Cyrus user since version 1.5.14 ;-)



Rob Siemborski wrote:
> 
> On Thu, 29 Jan 2004, Prentice Bisbal wrote:
> 
> > Proposed Design 2:
> > ------------------
> > Use 3 servers set up in a Murder aggregate as described
> > http://asg.web.cmu.edu/cyrus/ag.html. It is believed that this will
> > provide some failover protection, but I disagree. Again, the files will
> > be stored on the SAN.
> 
> This depends on what you mean by "some failover protection".  (Also what
> you mean by "3 servers"  -- 3 frontends, 3 backends, etc?)
> 
> It makes the failure of some part of the system more likely (more pieces
> make things more likely to break).  However, in the event of a failure,
> you will only loose a portion of the system (provided, of course, you
> don't mean 1 frontend, 1 backend, and 1 mupdate server).
> 
> -Rob
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456
> Research Systems Programmer * /usr/contributed Gatekeeper