Some part of bug from 2.3.4 still in 2.3.5

[Bron Gondwana]

And again...

[this message is brought to you by the "my god, I'm coding in C
 again, I'm too old for this shit" department]

... it appears reconstruct just blindly re-applied the value of
modseq from the already existing entry without checking if it
was valid.

The attached patch contains my previous patch to fix the
append_copy code path along with an additional check in
reconstruct which will set modseq to 1 if it doesn't 
already have a value.  Since modseq always starts at 1 and
is increased each operation, I think that's OK for people
running in the CONDSTORE universe as well.  It's certainly
fine for those of us where modseq will always be 1.

Bron.


On Thu, 25 May 2006 17:16:17 +1000, "Bron Gondwana" <brong at fastmail.fm> said:
> I suspect what you're looking for is in the patch attached
> to this message.
> 
> You forgot (entirely) to put modseq code in the append_copy
> path.  Since modseq appears to be a property of the target
> folder rather than the message itself, I'm using the same
> code as the 2.3.4->2.3.5 diff put into the other two append
> pathways.
> 
> I've rolled this out on to the server we're testing on and
> I've found that copy/move works.  Now to go reconstruct all
> the folders on that machine!
> 
> Bron.
> 
> 
> On Thu, 25 May 2006 16:50:05 +1000, "Robert Mueller" <robm at fastmail.fm>
> said:
> > Hi Ken
> > 
> > There's still a serious bug in 2.3.5 that causes copied messages to be 
> > pseudo invisible in the moved to folder. I can reproduce it as follows:
> > 
> > . select inbox.Archive
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected 
> > medeleted hasatt)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt 
> > selected medeleted hasatt \*)]
> > * 1 EXISTS
> > * 0 RECENT
> > * OK [UIDVALIDITY 1148525506]
> > * OK [UIDNEXT 185]
> > * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
> > . OK [READ-WRITE] Completed
> > 
> > . uid search 1:*
> > * SEARCH 84
> > . OK Completed (1 msgs in 0.000 secs)
> > 
> > So Archive folder has 1 message.
> > 
> > . select inbox
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected
> > hasatt 
> > NonJunk Junk medeleted)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt 
> > selected hasatt NonJunk Junk medeleted \*)]
> > * 13 EXISTS
> > * 0 RECENT
> > * OK [UNSEEN 12]
> > * OK [UIDVALIDITY 1148525506]
> > * OK [UIDNEXT 246932]
> > * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
> > . OK [READ-WRITE] Completed
> > 
> > . uid search 1:*
> > * SEARCH 180517 219479 227085 227785 228034 233668 238794 245170 245693 
> > 246382 246402 246930 246931
> > . OK Completed (13 msgs in 0.000 secs)
> > 
> > . uid copy 180517 inbox.Archive
> > . OK [COPYUID 1148525506 180517 185] Completed
> > 
> > So copy 1 message from inbox to Archive folder:
> > 
> > . select inbox.Archive
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt selected 
> > medeleted hasatt)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen hasnoatt 
> > selected medeleted hasatt \*)]
> > * 2 EXISTS
> > * 1 RECENT
> > * OK [UIDVALIDITY 1148525506]
> > * OK [UIDNEXT 186]
> > * OK [NOMODSEQ] Sorry, modsequences have not been enabled on this mailbox
> > . OK [READ-WRITE] Completed
> > 
> > . uid search 1:*
> > * SEARCH 84 185
> > . OK Completed (2 msgs in 0.000 secs)
> > 
> > Message appears in search index.
> > 
> > . uid fetch 185 rfc822
> > . OK Completed (0.000 sec)
> > 
> > Any attempt to retrieve message body data returns nothing :(
> > 
> > Rob
> > 
> -- 
>   Bron Gondwana
>   brong at fastmail.fm
> 
-- 
  Bron Gondwana
  brong at fastmail.fm

-------------- next part --------------
diff -ur cyrus-imapd-2.3.5/imap/append.c cyrus-imapd-2.3.5-modseq2/imap/append.c
--- cyrus-imapd-2.3.5/imap/append.c	2006-05-24 14:37:42.000000000 -0400
+++ cyrus-imapd-2.3.5-modseq2/imap/append.c	2006-05-25 03:39:14.000000000 -0400
@@ -857,6 +857,11 @@
     for (msg = 0; msg < nummsg; msg++) {
 	zero_index(message_index[msg]);
 	message_index[msg].uid = append_mailbox->last_uid + 1 + as->nummsg;
+	if (append_mailbox->options & OPT_IMAP_CONDSTORE) {
+	    message_index[msg].modseq = append_mailbox->highestmodseq + 1;
+	} else {
+	    message_index[msg].modseq = 1;
+	}
 	message_index[msg].last_updated = time(0);
 	message_index[msg].internaldate = copymsg[msg].internaldate;
 	as->nummsg++;
diff -ur cyrus-imapd-2.3.5/imap/reconstruct.c cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c
--- cyrus-imapd-2.3.5/imap/reconstruct.c	2006-03-31 14:22:27.000000000 -0500
+++ cyrus-imapd-2.3.5-modseq2/imap/reconstruct.c	2006-05-25 03:39:14.000000000 -0400
@@ -714,6 +714,10 @@
 	    /* Use data in old index file, subject to validity checks */
 	    message_index.internaldate = old_index.internaldate;
 	    message_index.modseq = old_index.modseq;
+	    /* This should never happen, but bugs in 2.3.4 and 2.3.5
+	     * could have left modseq blank.  If so, update it */
+	    if (!message_index.modseq)
+		message_index.modseq = 1;
 	    message_index.system_flags = old_index.system_flags &
 	      (FLAG_ANSWERED|FLAG_FLAGGED|FLAG_DELETED|FLAG_DRAFT);
 	    for (i = 0; i < MAX_USER_FLAGS/32; i++) {