<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Anthony,<br>
I think this is a different error, just hitting the same assertion
tripwire. I'm getting a problem in reconstruct(8), but your is
showing up in imapd.<br>
-nic<br>
<br>
<div class="moz-cite-prefix">On 07/26/2018 04:16 AM, Anthony Prades
via Cyrus-devel wrote:<br>
</div>
<blockquote type="cite"
cite="mid:e91a881b-e510-c1aa-c9c6-20cf27128dd3@chezouam.net">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<p>Hi,</p>
<p>Same problem here.</p>
<p>Context:<br>
Upgrade cyrus from 2.4.20 to Cyrus 3.0.7.<br>
As our spool is very big, we plan to run reconstruct -V max in a
second time as it seems to be possible from 2.5 (<a
class="moz-txt-link-freetext"
href="https://www.cyrusimap.org/2.5/imap/release-notes/2.5/x/2.5.0.html"
moz-do-not-send="true">https://www.cyrusimap.org/2.5/imap/release-notes/2.5/x/2.5.0.html</a>).</p>
<p>After upgrade binaries, mailbox are available until we change a
flag.<br>
After changing a flag, mailbox is unavailable with message:<br>
Jul 25 12:39:28 bluemind-debian cyrus/imap[9875]: Fatal error:
Internal error: assertion failed: imap/message.c: 4286:
!message_need(m, M_RECORD)<br>
Jul 25 12:39:28 bluemind-debian cyrus/master[8463]: process
type:SERVICE name:imap path:/usr/lib/cyrus/imapd age:0.138s
pid:9875 exited, status 75<br>
</p>
<p>To reproduce:<br>
- create a new mailbox using Cyrus 2.4.20: <a
class="moz-txt-link-abbreviated"
href="mailto:user/admin@apr-vmnet.loc" moz-do-not-send="true">user/admin@apr-vmnet.loc</a><br>
- put at least 2 messages - using LMTP:<br>
# ls -l
/var/spool/cyrus/data/mail/domain/a/apr-vmnet.loc/a/user/admin/<br>
total 12<br>
-rw------- 1 cyrus mail 1423 Jul 25 12:31 1.<br>
-rw------- 1 cyrus mail 1425 Jul 25 12:31 2.</p>
<p> - upgrade to cyrus 3.0.7<br>
- update flags on mail 1:<br>
# telnet localhost 143<br>
Trying 127.0.0.1...<br>
Connected to localhost.<br>
Escape character is '^]'.<br>
* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN
SASL-IR] server ready<br>
A1 LOGIN <a class="moz-txt-link-abbreviated"
href="mailto:admin@apr-vmnet.loc" moz-do-not-send="true">admin@apr-vmnet.loc</a>
admin<br>
A1 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten
QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME
UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH
SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID
THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE
ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS
LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE
SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH
URLAUTH=BINARY LOGINDISABLED X-QUOTA=STORAGE X-QUOTA=MESSAGE
X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User
logged in
SESSIONID=<cyrus-8408-1532514768-1-16031947181488935011><br>
A2 SELECT INBOX<br>
* 2 EXISTS<br>
* 0 RECENT<br>
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen)<br>
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen
\*)] Ok<br>
* OK [UNSEEN 1] Ok<br>
* OK [UIDVALIDITY 1532514648] Ok<br>
* OK [UIDNEXT 3] Ok<br>
* OK [HIGHESTMODSEQ 3] Ok<br>
* OK [URLMECH INTERNAL] Ok<br>
* OK [ANNOTATIONS 65536] Ok<br>
A2 OK [READ-WRITE] Completed<br>
A3 STORE 1 +FLAGS \Flagged<br>
* 1 FETCH (FLAGS (\Flagged))<br>
A3 OK Completed<br>
A4 LOGOUT<br>
* BYE LOGOUT received<br>
A4 OK Completed<br>
Connection closed by foreign host.<br>
</p>
<p> - select INBOX:<br>
# telnet localhost 143<br>
Trying 127.0.0.1...<br>
Connected to localhost.<br>
Escape character is '^]'.<br>
* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=PLAIN
SASL-IR] server ready<br>
A1 LOGIN <a class="moz-txt-link-abbreviated"
href="mailto:admin@apr-vmnet.loc" moz-do-not-send="true">admin@apr-vmnet.loc</a>
admin<br>
A1 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten
QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME
UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH
SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID
THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE
ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS
LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE
SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH
URLAUTH=BINARY LOGINDISABLED X-QUOTA=STORAGE X-QUOTA=MESSAGE
X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User
logged in
SESSIONID=<cyrus-8409-1532514786-1-17619737167055332223><br>
A2 SELECT INBOX<br>
* BYE Fatal error: Internal error: assertion failed:
imap/message.c: 4286: !message_need(m, M_RECORD)<br>
Connection closed by foreign host.<br>
</p>
<p>Running reconstruct -V max on this mailbox fix the problem but
we loose mail 2 flags as it is rediscovered:<br>
# reconstruct -V max <a class="moz-txt-link-abbreviated"
href="mailto:user/admin@apr-vmnet.loc" moz-do-not-send="true">user/admin@apr-vmnet.loc</a><br>
apr-vmnet.loc!user.admin: update uniqueid from header (null)
=> 5d0161495b585158<br>
apr-vmnet.loc!user.admin uid 2 rediscovered - appending<br>
apr-vmnet.loc!user.admin updating quota_mailbox_used: 4273 =>
2848<br>
apr-vmnet.loc!user.admin: updating exists 3 => 2<br>
apr-vmnet.loc!user.admin: updating sync_crc 3520663294 =>
4247682740<br>
<a class="moz-txt-link-abbreviated"
href="mailto:user/admin@apr-vmnet.loc" moz-do-not-send="true">user/admin@apr-vmnet.loc</a><br>
Repacked <a class="moz-txt-link-abbreviated"
href="mailto:user/admin@apr-vmnet.loc" moz-do-not-send="true">user/admin@apr-vmnet.loc</a>
to version 13</p>
<p># ls -l
/var/spool/cyrus/data/mail/domain/a/apr-vmnet.loc/a/user/admin/<br>
total 16<br>
-rwxr-x--- 1 cyrus mail 1423 Jul 25 12:31 1.<br>
-rwxr-x--- 1 cyrus mail 1425 Jul 25 12:31 3.</p>
<p>Anthony<br>
</p>
<div class="moz-cite-prefix">On 07/24/2018 07:10 PM, Nic Bernstein
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:b8988913-8602-540b-b01c-0e9bf14f37f6@onlight.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
Bron, et al.,<br>
Was this change ever cherry-picked to 3.0? I am seeing the same
issue with recent 3.0 HEAD, but slightly different location:<br>
<blockquote>user.masked: updating sync_crc 521983118 =>
503807715<br>
fatal error: Internal error: assertion failed: imap/message.c:
4286: !message_need(m, M_RECORD)<br>
</blockquote>
A git log of imap/message.c doesn't show a commit from 1/2/2017,
and nothing affecting imap/message.c around that time seems to
line up with this.<br>
<br>
Please advise,<br>
-nic<br>
<br>
<div class="moz-cite-prefix">On 01/02/2017 07:13 AM, Bron
Gondwana via Cyrus-devel wrote:<br>
</div>
<blockquote type="cite"
cite="mid:1483362793.2061068.834933537.7E80E6A9@webmail.messagingengine.com">
<title></title>
<div style="font-family:Arial;">Thanks for the data. It was 8
bytes of zeros across a UID and INTERNALDATE in the
cyrus.index file.<br>
</div>
<div style="font-family:Arial;"><br>
</div>
<div style="font-family:Arial;">I now have a fixed reconstruct
which can detect and repair this rather than aborting,
pushed to master.<br>
</div>
<div style="font-family:Arial;">I also have a Cassandane
testcase for this and a couple of other things that
reconstruct does :)<br>
</div>
<div style="font-family:Arial;"><br>
</div>
<div style="font-family:Arial;">Bron.<br>
</div>
<div><br>
</div>
<div>On Thu, 29 Dec 2016, at 09:45, Bron Gondwana via
Cyrus-devel wrote:<br>
</div>
<blockquote type="cite">
<div style="font-family:Arial;">Wow, interesting. Are you
willing to send me a tarball containing the spool folder
including cyrus.index and cyrus.cache files as well as the
email files themselves? I'll need your imapd.conf file as
well :)<br>
</div>
<div style="font-family:Arial;"><br>
</div>
<div style="font-family:Arial;">Cheers,<br>
</div>
<div style="font-family:Arial;"><br>
</div>
<div style="font-family:Arial;">Bron.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>On Thu, 29 Dec 2016, at 00:28, Thomas Cataldo via
Cyrus-devel wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div style="font-family:Arial;">Hi,<br>
</div>
<div><br>
</div>
<div>Running a build of 3.0.0-beta6 I hit the following
assertion on one of my test mailboxes after playing a
bit with the replication stuff :<br>
</div>
<div><br>
</div>
<div>
<p><span>root@bm1604:~#
/usr/lib/cyrus/sbin/sync_client -n eclipse -o -u <a
class="moz-txt-link-abbreviated"
href="mailto:tom@ex2016.vmw"
moz-do-not-send="true">tom@ex2016.vmw</a></span><br>
</p>
<p><span>Fatal error: Internal error: assertion
failed: imap/message.c: 4246: !message_need(m,
M_RECORD)</span><br>
</p>
<p><span>root@bm1604:~# cyradm -u admin0 localhost</span><br>
</p>
<p><span>Password: </span><br>
</p>
<p><span>localhost> version</span><br>
</p>
<p><span>name : Cyrus IMAPD</span><br>
</p>
<p><span>version : 3.0.0-beta6-3-gf721e5b</span><br>
</p>
<p><span>vendor : Project Cyrus</span><br>
</p>
<p><span>support-url: <a
href="http://www.cyrusimap.org"
moz-do-not-send="true">http://www.cyrusimap.org</a></span><br>
</p>
<p><span>os : Linux</span><br>
</p>
<p><span>os-version : 4.4.0-57-generic</span><br>
</p>
<p><span>environment: Built w/Cyrus SASL 2.1.26</span><br>
</p>
<p><span> Running w/Cyrus SASL 2.1.26</span><br>
</p>
<p><span> Built w/OpenSSL 1.0.2g 1 Mar
2016</span><br>
</p>
<p><span> Running w/OpenSSL 1.0.2g 1 Mar
2016</span><br>
</p>
<p><span> Built w/zlib 1.2.8</span><br>
</p>
<p><span> Running w/zlib 1.2.8</span><br>
</p>
<p><span> CMU Sieve 2.4</span><br>
</p>
<p><span> mmap = shared</span><br>
</p>
<p><span> lock = fcntl</span><br>
</p>
<p><span> nonblock = ioctl</span><br>
</p>
<p><span> idle = idled</span><br>
</p>
<p><br>
</p>
<p>root@bm1604:~# telnet localhost 1143<br>
</p>
<p>Connected to localhost.<br>
</p>
<p><span>Escape character is '^]'.</span><br>
</p>
<p><span>* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE
STARTTLS AUTH=PLAIN SASL-IR] server ready</span><br>
</p>
<p><span>. login <a class="moz-txt-link-abbreviated"
href="mailto:tom@ex2016.vmw"
moz-do-not-send="true">tom@ex2016.vmw</a> xxxxxx</span><br>
</p>
<p><span>. OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE
ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE
UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN
MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH
SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY
SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES
THREAD=REFS ANNOTATEMORE ANNOTATE-EXPERIMENT-1
METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS
LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE
SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1
X-REPLICATION URLAUTH URLAUTH=BINARY LOGINDISABLED
COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE
X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS
IDLE] User logged in
SESSIONID=<cyrus-43098-1482931179-1-18150325414286070410></span><br>
</p>
<p><span>. select inbox</span><br>
</p>
<p><span>* BYE Fatal error: Internal error: assertion
failed: imap/message.c: 4246: !message_need(m,
M_RECORD)</span><br>
</p>
<p><span>Connection closed by foreign host.</span><br>
</p>
<p><span></span><br>
</p>
<p><span>Trying to reconstruct the mailbox does not
help :</span><br>
</p>
<p><span>root@bm1604:~#
/usr/lib/cyrus/sbin/reconstruct -rfxGROU <a
class="moz-txt-link-abbreviated"
href="mailto:tom@ex2016.vmw"
moz-do-not-send="true">tom@ex2016.vmw</a></span><br>
</p>
<p><span><a class="moz-txt-link-abbreviated"
href="mailto:tom@ex2016.vmw"
moz-do-not-send="true">tom@ex2016.vmw</a></span><br>
</p>
<p>The error is still here after that.<br>
</p>
<p>Any idea ?<br>
</p>
<p>Regards,<br>
</p>
<p>Thomas.<br>
</p>
<p><br>
</p>
</div>
</div>
</blockquote>
<div style="font-family:Arial;"><br>
</div>
<div>
<div>--<br>
</div>
<div> Bron Gondwana<br>
</div>
<div> <a class="moz-txt-link-abbreviated"
href="mailto:brong@fastmail.fm" moz-do-not-send="true">brong@fastmail.fm</a><br>
</div>
<div><br>
</div>
</div>
<div style="font-family:Arial;"><br>
</div>
</blockquote>
<div style="font-family:Arial;"><br>
</div>
<div id="sig567075">
<div class="signature">--<br>
</div>
<div class="signature"> Bron Gondwana<br>
</div>
<div class="signature"> <a
class="moz-txt-link-abbreviated"
href="mailto:brong@fastmail.fm" moz-do-not-send="true">brong@fastmail.fm</a><br>
</div>
<div class="signature"><br>
</div>
</div>
<div style="font-family:Arial;"><br>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Nic Bernstein <a class="moz-txt-link-abbreviated" href="mailto:nic@onlight.com" moz-do-not-send="true">nic@onlight.com</a>
Onlight, Inc. <a class="moz-txt-link-abbreviated" href="http://www.onlight.com" moz-do-not-send="true">www.onlight.com</a>
6525 W Bluemound Road, Suite 24 v. 414.272.4477
Milwaukee, Wisconsin 53213-4073
</pre>
</blockquote>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Nic Bernstein <a class="moz-txt-link-abbreviated" href="mailto:nic@onlight.com">nic@onlight.com</a>
Onlight, Inc. <a class="moz-txt-link-abbreviated" href="http://www.onlight.com">www.onlight.com</a>
6525 W Bluemound Road, Suite 24 v. 414.272.4477
Milwaukee, Wisconsin 53213-4073
</pre>
</body>
</html>