Recomendations for a 15000 Cyrus Mailboxes

[Greg A. Woods]

At Thu, 12 Apr 2007 10:37:56 +1000, Bron Gondwana wrote:
Subject: Re: Recomendations for a 15000 Cyrus  Mailboxes
> 
> Agree on the external controllers.  Ours are SCSI attached.  We haven't
> tried LVM, but the separation of responsibilities means a kernel crash
> on the OS won't take out the RAID system and vice versa.

Exactly.  A good SAN or SAN-like storage array, even if it is only
attached to one host system, makes many problems simply go away
completely.

> Multiple systems!  Also good for denial of service protection, making
> really sure your 'admin' user is blocked for external users and a few
> other funky things.

I haven't yet seen any kind of DoS for IMAP or IMAP/s against the
systems I've run, though I've sometimes worried about too many of those
clients which can tend to open many connections for power users.  So far
so good though.  I don't worry about the admin user being blocked
either.  In fact that would never be a worry for me -- I simply don't
rely on that level of access for control over the system(s).

> As for complexity?  It's on the cusp.  We've certainly had many more
> users on a single instance before, but we prefer to keep under 10k users
> per Cyrus instance these days for quicker recoverability.  It really
> depends if this system is expected to grow.  Once you build a management
> system that allows you to run more than one Cyrus instance painlessly,
> you can scale out to multiple machines much more easily.

As you say, it really does depend on how the system is to grow, and how
fast it is to grow.  I wouldn't even dream of growing that complexity
unless I expected to grow to well over 50,000 users these days (and,
say, 150k mailboxes, which is generous for the types of users I've
typically dealt with on installations of this size).  I expect hardware
capabilities to increase to allow for doubling that peak size again much
quicker than the average user base will grow even in a small-ISP
scenario (unless they get into acquisitions :-)).

The only concern I would have is if my users expected higher than
deliverable availability.  I.e. if my hardware support contracts
couldn't deliver a new running identically configured box before the SLA
for the users ran out then I too would want at least an N+1
configuration so that any dead box could be worked around quickly.
However the costs of operating a multi-server configuration are a _lot_
higher, never mind the also much higher cost of designing and building
one, even with all the help the base Cyrus software gives.  A little bit
of expectation management can go a long way towards getting the hardware
support and user SLAs to meet in the middle too.

Note that the same goes for the SAN as well.  Both pieces of the puzzle,
computing and storage, must either be designed for HA and 100% reliable
operation, or be quickly and easily replaced.  Typically computing
machinery is easier to replace and SAN boxes are typically easier to
design for HA operation with redundant hot-swappable bits everywhere.

Even in an ISP scenario the hardware support contracts, and the high end
hardware, are cheaper than the operating costs of having sufficiently
skilled full-time operations staff.  At least in North America so far as
I've seen.  (and finding the silled people could be another trick,
unless you can also afford to train them from scratch, and possibly keep
doing that as they get bored and want to move on)

For a mere 15,000 users, and growth capacity to double that in, say,
another two to five years, it would be ludicrous to use more than one
Cyrus server, unless maybe your users were some 24x7 high-demand group,
like a hospital or police/fire/emerg centre, etc.  Even a pair of
dual-core MX servers would be over-kill unless the user base was a known
high-risk DoS target too, but I'd still keep the MX separate just so
that adding another MX host would be a half-hour job for a junior baby
sysadmin at worst.

-- 
						Greg A. Woods

H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods at robohack.ca>
Planix, Inc. <woods at planix.com>       Secrets of the Weird <woods at weird.com>