sync_client stalls the rest of cyrus while 'no route to host'

[Bron Gondwana]

On Tue, 29 Aug 2006 09:51:49 +0100 (BST), "David Carter" <dpc22 at cam.ac.uk> said:
> On Sun, 27 Aug 2006, Bron Gondwana wrote:
> 
> > To tell you the truth, I'm seriously considering writing a replacement to
> > sync_client that does a bunch of different things including multiple
> > replicas, maintaining log files, etc.  All of this drops out pretty easily
> > from a pattern which produces a single log file per day and calls the
> > sync_client fork children with a byte-range on the log file to run rather
> > than moving the file and then running the copy.
> 
> I think that you would be better off with multiple log files and multiple
> sync_client processes, one for each replica. That way each replication
> stream is independent and can progress at its own best speed.
> 
> Particularly important if a replica dies (or is shut down for routine 
> maintenance) and needs to catch up from a big backlog of transactions.

I fail to see the difference between this and a single big log file where
each replica keeps track of where it's up to in the log file and then seeks
to that spot, reads up to say 10,000 lines and processes that, then stores
the new location so they next process can seek directly there before starting.

This pattern means a lot less copying data to multiple log files and deleting
again.

It certainly doesn't mean every replica needs to be at the same point in the
log file!

This is how mysql handles replication - with each replica keep track of two
values - which log file it's in (they're numbered sequentially starting from
1, and going up to wherever you get to before you reset replication for some
reason) and how many bytes through that file it is.

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm