sync_client stalls the rest of cyrus while 'no route to host'

[David S. Madole]

> From David Carter on Tuesday, August 29, 2006 4:52 AM:
> 
> On Sun, 27 Aug 2006, Bron Gondwana wrote:
> 
> > To tell you the truth, I'm seriously considering writing a 
> replacement 
> > to sync_client that does a bunch of different things including 
> > multiple replicas, maintaining log files, etc.  All of this 
> drops out 
> > pretty easily from a pattern which produces a single log 
> file per day 
> > and calls the sync_client fork children with a byte-range 
> on the log 
> > file to run rather than moving the file and then running the copy.
> 
> I think that you would be better off with multiple log files 
> and multiple sync_client processes, one for each replica. 
> That way each replication stream is independent and can 
> progress at its own best speed.
> 
> Particularly important if a replica dies (or is shut down for routine
> maintenance) and needs to catch up from a big backlog of transactions.

I have written some code that implements something much like you describe as a separate daemon outside of Cyrus itself that is an intermediary between the Cyrus sync log file and sync_client. It creates multiple copies of the sync log for each replica and fires off per-replica instances of sync_client at intervals to process replications to multiple replica servers.

I've been running it for a while in a setup with two replicas on a small number of test user mailboxes and it seems to do the right thing, but I am probably not really ready yet to rely on  it (or perhaps replication as implemented in 2.3.x at all) for larger populations of users.

However, since recent conversations here have been on a similar topic, I would like to offer it for comments and discussion and in the event that it may be useful to someone. It is available at http://madole.net/pushmail/

David