<div dir="ltr"><div>Hi Nic,</div><div>as far as I know there are currently a couple of nasty open issues when upgrading old mailboxes, created from earlier versions of cyrus, that have not been upgraded to v13 index using reconstruct. These issues are:</div><div><br></div><div><a href="https://github.com/cyrusimap/cyrus-imapd/issues/2171">https://github.com/cyrusimap/cyrus-imapd/issues/2171</a></div><div><a href="https://github.com/cyrusimap/cyrus-imapd/issues/2208">https://github.com/cyrusimap/cyrus-imapd/issues/2208</a></div><div><br></div><div>The main problem is frequent crashes with error message "assertion failed" when accessing these mailboxes.</div><div><br></div><div>In issue #2208 a reconstruct command was proposed as a workaround:</div><div>
<code>reconstruct -G -V max [mailboxname]</code> <br></div><div><br></div><div>Note that -G parameter seems necessary, otherwise the problem persists...</div><div><br></div><div>Note also, that this should be done while still in version 2.5.x! The behavior of version 3.x reconstruct is not the same and you will not have the same results.</div><div><br></div><div>As for the cyrdump you mention, it would be better if you compared the results before running reconstruct and after that.</div><div> I guess what you would see is that the uidlist before would probably be in the range 6 - 9 (or you'd see a crash). cyrus 3.0.x considers that part of the index corrupt so simply cannot access those uids (1-5). Reconstruct detects this issue and what it does is that <b>reappends </b>those "newly found" messages, giving them a new uid, so that's what uids 10-14 probably really are. Note that this causes these messages to appear unread, since they have no flags (not something you want for your clients' mailboxes).</div><div><br></div><div>Bron mentioned something about this bug in the cyrus devel list a few days ago:</div><div><a href="https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2018-July/thread.html#4313">https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2018-July/thread.html#4313</a><br></div><div>but I didn't see any related commit or anything mentioned in the above issues, so I'm not sure if there is a fix at the moment...</div><div><br></div><div>Hope this helps,</div><div>Regards,</div><div>Savvas Karagiannidis<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jul 31, 2018 at 6:43 PM Nic Bernstein <<a href="mailto:nic@onlight.com">nic@onlight.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
Friends,<br>
I'm preparing for a couple of belated 2.5.X to 3.0.X upgrades, and
have a question about how necessary it is to run "reconstruct -V
max" on the mailstore. Both systems are currently running 2.5.10,
and are already at index version 13. However, when performing
"reconstruct -V max" on one, on a new 3.0.7 (with patches) system, I
see this:<br>
<blockquote>
<pre>root@newmail:~# /usr/lib/cyrus/bin/reconstruct -V max user/onlight
user.onlight uid 1 rediscovered - appending
user.onlight uid 2 rediscovered - appending
user.onlight uid 3 rediscovered - appending
user.onlight uid 4 rediscovered - appending
user.onlight uid 5 rediscovered - appending
user/onlight
Repacked user/onlight to version 13</pre>
</blockquote>
The last line can be ignored, as it's really a noop. The
"rediscovered - appending" stuff is what catches my eye. However,
once the reconstruct is complete, here's what the mailbox looks
like:<br>
<blockquote>
<pre>root@newmail:/var/spool/cyrus/mail/I/user/onlight# /usr/lib/cyrus/bin/cyrdump user/onlight
Content-Type: multipart/related; boundary="dump-27466-1533049817-351841533"
--dump-27466-1533049817-351841533
Content-Type: text/xml
IMAP-Dump-Version: 0
<imapdump uniqueid="710a47ca47ebc676">
<mailbox-url>imap://<a href="http://newmail.example.com/user.onlight" target="_blank">newmail.example.com/user.onlight</a></mailbox-url>
<incremental-uid>0</incremental-uid>
<nextuid>15</nextuid>
<b> <uidlist>6 7 9 10 11 12 13 14 </uidlist></b>
<flags>
...
</pre>
</blockquote>
Note that the <uidlist> doesn't list those low number UIDs
which were listed in the reconstruct sequence. In other words, I
think this all is harmless, but I'm not sure how much overhead it
brings to the whole process. <br>
<br>
One of the servers has a total of 70GB of mail, so a complete
reconstruct run only takes a short while. The other, however, has
over 8TB scattered over >30 partitions. If I could avoid running
reconstruct across that whole wad, it'd be great.<br>
<br>
Thoughts please?<br>
-nic<br>
<pre class="m_4724969770412079391moz-signature" cols="72">--
Nic Bernstein <a class="m_4724969770412079391moz-txt-link-abbreviated" href="mailto:nic@onlight.com" target="_blank">nic@onlight.com</a>
Onlight, Inc. <a class="m_4724969770412079391moz-txt-link-abbreviated" href="http://www.onlight.com" target="_blank">www.onlight.com</a>
6525 W Bluemound Road, Suite 24 v. 414.272.4477
Milwaukee, Wisconsin 53213-4073
</pre>
</div>
----<br>
Cyrus Home Page: <a href="http://www.cyrusimap.org/" rel="noreferrer" target="_blank">http://www.cyrusimap.org/</a><br>
List Archives/Info: <a href="http://lists.andrew.cmu.edu/pipermail/info-cyrus/" rel="noreferrer" target="_blank">http://lists.andrew.cmu.edu/pipermail/info-cyrus/</a><br>
To Unsubscribe:<br>
<a href="https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus" rel="noreferrer" target="_blank">https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus</a></blockquote></div>