<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 07/24/2015 12:07 AM, ellie timoney wrote:<br>
<blockquote
cite="mid:1437714474.3986135.331859313.0D8AD1EF@webmail.messagingengine.com"
type="cite">
<title></title>
<blockquote>
<blockquote>
<blockquote type="cite">
<div>- a single cyrus instance may be the primary server for
some users but a
replica server for other users<br>
</div>
</blockquote>
<div>Are you sure about that? <br>
</div>
</blockquote>
<div> </div>
<div>I'm not sure about any of this. But you make a good point:
I thought I could see a way that this was possible, but now I
don't think I can.<br>
</div>
</blockquote>
<div> </div>
<div>I think I can, again, though I guess not in a murder.<br>
</div>
<div> </div>
<div>But in a non-murder setup, I believe the only difference
between a primary vs replica is "which one the user interacts
with"? Which, in a non-murder setup, one is presumably managing
in some way external to cyrus (e.g. database, proxies, and some
glue)...<br>
</div>
<div> </div>
<div>So it seems like it should be plausible to have two (or more)
cyrus instances (hosted wherever), each running a sync_server
plus a channel/sync_client for each of the others, and whatever
happens on any replicates to the others. You'd be trusting your
outer, non-cyrus layer to make sure not to proxy any individual
user to the wrong instance (at least, not while there's pending
replication transactions for them). And I guess shared folders
would be thorny. But I don't (yet) see a reason from a
replication standpoint why this wouldn't work.<br>
</div>
<div> </div>
<div>So:<br>
</div>
<div> </div>
<div>- a single cyrus instance may* be the primary server for some
users but a
replica server for other users</div>
<div> </div>
<div>[* with caveats and requiring custom tooling, and
specifically not in a murder]<br>
</div>
</blockquote>
<br>
Yikes, well I guess we can do pretty much anything*. :-0 But, I
think, in the context of Nicola's original post, it's safer to say
that this is not a standard or supported configuration. Nicola is
writing documentation, so an understanding of typical usage trumps
speculative possibilities. :-)<br>
<br>
I do think, however, that a more thorough description of the roles
of sync_client(s) and sync_server are in order, and if I have a
chance will write something up soon.<br>
<br>
Cheers,<br>
-nic<br>
<br>
*PS - Imagine the "sync storm" resulting from such an arrangement
run amok!<br>
<br>
<blockquote
cite="mid:1437714474.3986135.331859313.0D8AD1EF@webmail.messagingengine.com"
type="cite">
<div> </div>
<div>ellie<br>
</div>
<div> </div>
<div>On Fri, Jul 24, 2015, at 10:01 AM, ellie timoney wrote:<br>
</div>
<blockquote type="cite">
<pre>
</pre>
<blockquote>
<div>Okay, I'll bite. Here's what a bit of a sync_log looks
like:<br>
</div>
</blockquote>
<div> </div>
<div>Thanks! So anything processing a sync_log (sync_client,
squatter, etc) needs to look at an actual copy of the
user/mailbox in order to determine its current state, and
needs to look at both copies to work out what to replicate
between them.<br>
</div>
<div> </div>
<blockquote>
<blockquote type="cite">
<div>- a single cyrus instance may be the primary server for
some users but a
replica server for other users<br>
</div>
</blockquote>
<div>Are you sure about that? <br>
</div>
</blockquote>
<div> </div>
<div>I'm not sure about any of this. But you make a good point:
I thought I could see a way that this was possible, but now I
don't think I can.<br>
</div>
<div> </div>
<div>Cheers,<br>
</div>
<div> </div>
<div>ellie<br>
</div>
<div> </div>
<div>On Thu, Jul 23, 2015, at 11:42 PM, Nic Bernstein wrote:<br>
</div>
<blockquote type="cite">
<div>On 07/23/2015 01:14 AM, ellie timoney wrote:<br>
</div>
<blockquote
cite="mid:1437632078.1806425.330925921.3A2B9A7D@webmail.messagingengine.com"
type="cite">
<blockquote type="cite" style="color:rgb(0, 0, 0);">
<pre>Is the file format of the sync log defined anywhere? I assume it
<span>> </span>correlates with a set of commands. (Not that this is important to a
<span>> </span>user: it may as well be opaque, but it made me wonder!)
</pre>
</blockquote>
<pre>I'm a bit confused about this myself. Each time I go digging into the
code my understanding flips back the opposite way.
I think, either:
* the sync log contains all the information needed to reproduce what's
happened (e.g. if a message has arrived, the sync log will contain the
message itself); OR
* the sync log contains just enough to identify things that have changed
(e.g. if a message has arrived, the sync log contains a message id of
some sort), and the sync_client processing the log just uses the log to
discover which things to sync, but then uses the actual mailbox to
construct the changes to send to the replica.
Either way I haven't seen any documentation on the sync log format. I
suspect it's either the raw sync protocol or some subset thereof?
</pre>
</blockquote>
<div> </div>
<div>Okay, I'll bite. Here's what a bit of a sync_log looks
like:<br>
</div>
<blockquote>
<pre>MAILBOX user.newjersey
MAILBOX user.support
USER onlight
USER nic
USER admin
USER randy
MAILBOX user.randy.Trash
USER lynn
MAILBOX user.lynn.Trash
</pre>
</blockquote>
<div>It's basically a list of either users or mailboxes which
have been altered. When sync_log is enabled, all of the
daemons which might alter a mailbox or user will write a
line to this log each time they do so. That means the
obvious suspects -- imapd, pop3d, timsieved, lmtpd, etc. --
but also cyr_expire and friends (as in the USER...MAILBOX
couplets at the end of the sample).<br>
</div>
<div> </div>
<blockquote type="cite">
<pre>- a single cyrus instance may be the primary server for some users but a
replica server for other users
</pre>
</blockquote>
<div>Are you sure about that? How does one specify the users
for which such an instance would play each role? A single
HOST may run separate instances, which may perform these
different roles, but I cannot fathom how to configure a
single instance to do both at once for different user
cohorts.<br>
</div>
<div> </div>
<div>This raises potential problems when one deploys
replication within a murder (Cyrus aggregation). Only one
server may claim ownership of any given mailbox, via a
mupdate call, so an instance which is a replica MAY NOT push
updates to mupdate master, or mayhem will ensue. Here's a
commented section from /etc/cyrus.conf on a replication
master instance:<br>
</div>
<pre>##
# Master sends mailbox updates to mupdate.
# Replication client runs on Master.
# Comment these 2 lines out on replicas
mupdatepush cmd="/usr/lib/cyrus/bin/ctl_mboxlist -m"
syncclient cmd="/usr/lib/cyrus/bin/sync_client -r"
</pre>
<div>A nice daydream of mine envisions a world wherein
mailboxes.db keeps track of replica locations, as well,
which would allow for the dual-role operation Ellie
describes.<br>
</div>
<div> </div>
<div>Cheers,<br>
</div>
<div> -nic<br>
</div>
<pre>--
Nic Bernstein <a moz-do-not-send="true" href="mailto:nic@onlight.com">nic@onlight.com</a>
Onlight llc. <a moz-do-not-send="true" href="http://www.onlight.com">www.onlight.com</a>
219 N. Milwaukee St., Ste. 2A v. 414.272.4477
Milwaukee, Wisconsin 53202 f. 414.290.0335
</pre>
</blockquote>
<div> </div>
</blockquote>
<div> </div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Nic Bernstein <a class="moz-txt-link-abbreviated" href="mailto:nic@onlight.com">nic@onlight.com</a>
Onlight, Inc. <a class="moz-txt-link-abbreviated" href="http://www.onlight.com">www.onlight.com</a>
1442 N Farwell Ave., Suite 600 v. 414.272.4477
Milwaukee, Wisconsin 53202
</pre>
</body>
</html>