cyrus replication validation

[Wesley Craig]

If I understand this patch correctly, it doesn't solve the larger  
problem that I'm interested in: is the data on my replica the same as  
the data on my primary, or more to the point, are the two data sets  
converging?  This patch *would* allow me to more or less validate  
that the cyrus.* meta files are more or less the same on both, and  
takes care of the sync_server staling staging issue -- whether the  
cyrus.* meta files match the other files.  But I'm really interested  
in something that can run out of band from csync, imap, etc, that  
examines files on the primary and replica to know what the variance  
is.  I think make_md5 is pretty ideal for what I'm after, as a source  
of data.  We're working on scripts that compare the data files.

:wes

On 06 Apr 2007, at 22:31, Rob Mueller wrote:
>> The provided Cyrus tool "make_md5" is for validating replication.  It
>> would, for instance, have found the recently discussed bug in   
>> sync_server that caused random files to be overwritten in the  
>> event  that sync_server reused a stale staging file.  It would  
>> probably be  cool if there were documentation somewhere that  
>> advised people on how  to run it and how to use it to validate  
>> replication.
>
> We have a patch that helps with this as well see "MD5 UUIDs" here:
>
> http://cyrus.brong.fastmail.fm/
>
> Basically it does two things:
> 1. You can make the UUIDs of all messages the first 11 bytes of the  
> MD5 of the message
> 2. You can fetch a computed MD5 of any message on disk via IMAP
>
> Using the second, you can do complete validation via IMAP, just  
> iterate through all folders and all messages, get the computed MD5  
> and compare on both sides.
>
> The UUID bit is just designed to help replication when messages are  
> moved between folders, rather than having to resend the entire  
> message on a move, it can just link them from one folder to the  
> other at the replication end.