Implement Cyrus IMAPD in High Load Enviromment
bawood at umich.edu
Wed Sep 30 01:01:28 EDT 2009
On Tuesday 29 September 2009 @ 18:41, Bron Gondwana wrote:
> Possibly the secret is that we use IPAddr2 from linux-ha to force
> ARP flushes, and we transfer the primary IP address between
> machines, so nothing else needs to know - we just shut down one end
> and bring up the other with the IP and it's all good.
Our primaries and replicas are located in different data centers, and
since we have not control over how the network is subdivided it's
impossible for them to take the same IPs.
> Our process is:
> a) check there are less than 10kb of files in $conf/sync/ - else
> abort b) shut down the master (host A)
> c) run sync_client -f $file on each file in $conf/sync (if any)
> c2) (if any sync fails, restart the master (host A))
> d) shut down the replica (host B)
> e) update the database with the new master location
> f) start up the replica (host A)
> g) start up the master (host B)
> This means replication starts immediately, because the replica is
> already there when the master starts.
So you just immediately start replicating back to a host (or site)
that just failed? How does that work?
We have a third level of machines that we sync to, in an out of band
process, but the data is stored exactly the same way so we can start
replicating to them immediately. So even if a entire data center
failed, we can still be running a fully replicated service with
almost no downtime visible to users.
More information about the Info-cyrus