replication, how to see if it 'up to date'

[Bron Gondwana]

On Thu, 28 Sep 2006 09:55:53 +0100 (BST), "David Carter" <dpc22 at cam.ac.uk> said:
> On Thu, 28 Sep 2006, Rudy Gevaert wrote:
> 
> > Does anybody know how to see if the sync replication is up to date?  All 
> > suggestions are welcome.
> 
> I replicate all users from all masters to replicas about once a month and 
> check the output for any inconsistancies. Something like:
> 
>    cyrus-23[cyrus:cyrus]$ replicate -s cyrus-24 -v -v -u dpc22
>    USER dpc22
>      USER_ALL dpc22
>      ENDUSER
> 
> where "replicate" is just a little wrapper around sync_client.
> 
> We also maintain databases of MD5 checksums for messages and cache 
> entries, generated by make_md5.

Our monitoring script that runs every 2 minutes does a "du -s" on the
{configdirectory}/sync directory and notifies us if it gets "big" (where
big is obviously a per-instance variable)

A less frequently run script searches the ps output for copies of
sync_client.  It then looks in they sync directory, and for any
file called log-{N} it checks if there is a sync_client with PID
{N}.  If not, it tries to run it with -r -f.  Finally, if there
was no sync_client at all, it launches a new one with -r.

Thirdly we have a tool which makes an imap connection to each of
the master and replica (we run an imapd on a different port on the
replica purely for this purpose) and does a bunch of checks
on each mailbox.  It has to run as each user to get SEEN state.
This script has multiple levels of test, right up to one that
actually randomly picks a few messages and fetches their bodies
for comparison.  It always checks at least flags and everything
you can STATUS for.  This tool also has an optional flag which
can ssh to the apropriate servers and run reconstruct on the
mailboxes as well as sync_client -u on the offending user if it
finds inconsistencies.  Obviously, there's a possibility that
it catches changes where sync_client just hasn't caught up yet,
but it retries multiple times with a tunable delay, so generally
it will get the same value on one of the retries unless the
folder is seeing a lot of traffic.

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm