Cyrus IMAP and MySQL mailboxes (Building load-balancing cluster)

[Igor Zhbanov]

2006/11/17, Sarah Walters <s.walters at its.uq.edu.au>:
> > -----Original Message-----
> > From: info-cyrus-bounces at lists.andrew.cmu.edu
> > [mailto:info-cyrus-bounces at lists.andrew.cmu.edu] On Behalf Of
> > Igor Zhbanov
> > Sent: Friday, 17 November 2006 10:32 AM
> > To: info-cyrus at lists.andrew.cmu.edu
> > Subject: Re: Cyrus IMAP and MySQL mailboxes (Building
> > load-balancing cluster)
> >
> > 2006/11/17, Sarah Walters <s.walters at its.uq.edu.au>:
> > > > > > The main problem is to build a shared storage that can
> > > > survive server
> > > > > > crashes, where mail will be stored.
> > > > >
> > > > > Use a SAN.
> > > > First of all, such SAN must be very reliable itself.
> > Second, it must
> > > > support some kind of global locking mechanism, so several
> > nodes can
> > > > use lock to protect file from simultaneous writing.
> > Third, Cyrus IMAP
> > > > must lock mailboxes, so several instances on different
> > server can work
> > > > with one mailbox without conflicts. Whether Cyrus IMAP
> > use locks or
> > > > assumes that he is the only one who access mail box, I
> > don't know. Can
> > > > it safely access one mailbox from different servers, I
> > don't know too.
> > >
> > > A good commercial SAN costs a fortune, but it is very
> > reliable. And did
> > > you want a *cluster* or a group of servers? A cluster
> > should operate as
> > > if it is a single host, and thus avoid the locking issues.
> > Have a look
> > > at Red Hat's clustering product for an example. If our cyrus
> > > installation was going to be clustered, RH cluster with a
> > SAN backend is
> > > what we would do.
> >
> > Yes, I need the cluster exactly. Have I lots of domains, I could store
> > mailboxes of each domain on separate server. But I have only one big
> > domain. So, I need to spread mailboxes on one domain across several
> > servers. And than I need very clever load-balancer that will send
> > request to server where particular mailbox is. I don't know such
> > balancers aware of mail system structure.
>
> I believe that Sun's mail server can do this, but if you lost a
> particular
> server you would lose the mailboxes on that server (or rather, would
> have
> to put up a new server connected to that part of the SAN storage).
>
> Why don't you look at throwing two beefy boxes at this problem in a
> hot-spare
> configuration? Have a single large box managing the mail and a heartbeat
> so
> that if one goes down the other immediately takes over its IP and just
> keeps
> going? You will lose anything that is actually in memory, but that
> shouldn't
> be an issue as long as you are using a SAN and immediately committing to
> disk
> rather than using a solution like MySQL. There is no need for load
> balancing
> here as far as I can tell, and what you lose in having to buy a chunkier
> server
> you will gain in reduced power consumption and associated data centre
> costs.

Yes, I know how failover cluster works. But what if one server
(active) can't process such a load? Suppose, we plan to have 100 000
users working actively with mail. I understand that it is possible to
use one monstrous server to take all of the load, but I am interested
in load-balancing solution on relatively inexpensive servers. And what
about slow anti-viruses for 100 000 users' mail? Or to use
load-balanced front-ends connected to single SAN and connected to
anti-virus load-balanced cluster? :-)

---

2006/11/17, Simon Matter <simon.matter at invoca.ch>:
> > 2006/11/17, Sarah Walters <s.walters at its.uq.edu.au>:
> >> > > > The main problem is to build a shared storage that can
> >> > survive server
> >> > > > crashes, where mail will be stored.
> >> > >
> >> > > Use a SAN.
> >> > First of all, such SAN must be very reliable itself. Second, it must
> >> > support some kind of global locking mechanism, so several nodes can
> >> > use lock to protect file from simultaneous writing. Third, Cyrus IMAP
> >> > must lock mailboxes, so several instances on different server can work
> >> > with one mailbox without conflicts. Whether Cyrus IMAP use locks or
> >> > assumes that he is the only one who access mail box, I don't know. Can
> >> > it safely access one mailbox from different servers, I don't know too.
> >>
> >> A good commercial SAN costs a fortune, but it is very reliable. And did
> >> you want a *cluster* or a group of servers? A cluster should operate as
> >> if it is a single host, and thus avoid the locking issues. Have a look
> >> at Red Hat's clustering product for an example. If our cyrus
> >> installation was going to be clustered, RH cluster with a SAN backend is
> >> what we would do.
> >
> > Yes, I need the cluster exactly. Have I lots of domains, I could store
> > mailboxes of each domain on separate server. But I have only one big
> > domain. So, I need to spread mailboxes on one domain across several
> > servers. And than I need very clever load-balancer that will send
> > request to server where particular mailbox is. I don't know such
> > balancers aware of mail system structure.
>
> The question here is what is "one big domain" exactly? How many users,
> what's the expected mailbox size (quota) and which usage pattern. That's
> what really matters here.

By one big domain I mean network domain. Consider gmail.com. You
cannot make load-balancing by domain names.
About 100 000 users with 100 mb mailboxes.

> > Another way is to try to use something like GFS, constructed from
> > several servers. Each server will contain one portion of data and will
> > have Cyrus IMAP instance. But GFS will be mounted to each server, so
> > each Cyrus IMAP instance will see whole filesystem. Of course, here we
> > have same possible lock issues as with NFS. And we have no redundancy,
> > as I understand.
> > So, I need cluster for reliability to. I want to have each mailbox at
> > least on two servers (or to have shared storage upon two-nodes
> > failover cluster), so I can survive crash of any node (and not only of
> > disk system, that can be protected by RAID, but CPU crash, memory...)
> > with minimal recovery time.
>
> You may also search the list archives for some more info on the topic.
> There has been alot of discussion recently.

Please, tell me few keywords for search. Cyrus + GFS?