<!DOCTYPE html>
<html>
<head>
<title></title>
<style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style>
</head>
<body><div style="font-family:Arial;"><br></div>
<div><br></div>
<div><br></div>
<div>On Mon, 26 Feb 2018, at 10:37, Sebastian Hagedorn wrote:<br></div>
<blockquote type="cite"><div>Hi,<br></div>
<div><br></div>
<div>we're only getting started with replication, so I can't contribute any<br></div>
<div>useful comments, but your post raised two questions for me:<br></div>
<div><br></div>
<div>This only affects the "new", post-3.0 replication protocol, right? We're<br></div>
<div>planning a migration from 2.4 to 3.0 using replication, and I'd like to be<br></div>
<div>aware of all potential issues we might encounter.<br></div>
</blockquote><div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">A migration will be totally fine regardless.<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">I realise now the danger of posting about a really rare edge case to the dev list in a way that implies that it's anything anyone is likely to hit in regular usage. It surfaced something like 3 times over tens of thousands of mailboxes and a full disk event causing some weirdness around retries in sync. Cyrus handles full disk quite well these days (after an event in 2014 where we fixed a bunch of bugs!) but it does make some things odd. In particular, it can cause replication to fail for a mailbox due to disk full, but then later mailboxes in the same list get processed OK because there was space for that operation.<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">In summary, don't let your disks get full!<br></div>
<div style="font-family:Arial;"><br></div>
<blockquote type="cite"><blockquote><div>Hey - here's me posting something to the public list instead of internal<br></div>
<div>FastMail slack. We've been really bad at making our random ruminations<br></div>
<div>public, sorry.<br></div>
<div>Tomorrow (Tues 27th) 2pm Melbourne time, I'm going to be meeting with<br></div>
<div>ellie and maybe Partha in the Melbourne office with a whiteboard and a<br></div>
<div>screen to flesh out some ideas for things we can do to fix some of the<br></div>
<div>issues that came up after a recent machine failure event at FastMail.<br></div>
</blockquote><div><br></div>
<div><snip><br></div>
<div><br></div>
<blockquote><div>b) there are over 1000 mailboxes, and the log file got deduplicated and<br></div>
<div> then run in sets, and we had this:<br></div>
<div>sync_log MAILBOX Z<br></div>
<div>sync_log MAILBOX B<br></div>
<div><br></div>
<div>(for a rename of Z to B)<br></div>
</blockquote><div><br></div>
<div>Where and when does deduplication happen?<br></div>
</blockquote><div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">Deduplication in this case is sync_action_list_add(mailbox_list, args[1]) in sync_client.c:do_sync().<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">If the same item is logged twice in the same log file, then it's only added once to the list of mailboxes later passed to do_mailboxes.<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;">Bron.<br></div>
<div style="font-family:Arial;"><br></div>
<div style="font-family:Arial;"><br></div>
<div id="sig56629417"><div class="signature">--<br></div>
<div class="signature"> Bron Gondwana, CEO, FastMail Pty Ltd<br></div>
<div class="signature"> brong@fastmailteam.com<br></div>
<div class="signature"><br></div>
</div>
<div style="font-family:Arial;"><br></div>
</body>
</html>