Problem moving between folders during 2.3.16 -> 2.5.9 upgrade

ellie timoney ellie at fastmail.com
Mon Jul 25 01:53:43 EDT 2016 

I don't have a 2.3.x environment handy to try to reproduce this from the
outside in, so I'm starting at the error code and working outwards...

> > <1469384361<a0080 UID COPY 1162729 "Drafts"
> > >1469384362>a0080 NO Mailbox format corruption detected

"Mailbox format corruption detected" corresponds with
IMAP_MAILBOX_CHECKSUM

This error code is exclusively returned by functions in imap/mailbox.c,
for index/record crc mismatches.  Given that the mailbox is selectable
and such, I'm supposing that the whole-file/whole-record crc checks are
okay...

It looks like 2.3.16 had MAILBOX_MINOR_VERSION 10, and in particular
doesn't have the record->cache_crc field, which many of the
IMAP_MAILBOX_CHECKSUM checks are for.  When reading mailboxes < 12 in
newer Cyrus versions, this field is initialised to zero.

Some of the places that want to do a crc check against record->cache_crc
first make sure record->cache_crc is non-zero before doing so: 
cache_append_record(), mailbox_cacherecord().

But some don't, or at least don't appear to: mailbox_append_cache()
initially uses cache_append_record() (good), but then reads it straight
back in to ensure freshness -- and then having done so, appears to do an
unconditional check against record->cache_crc (dodgy?)

Actually that seems like all the spots this check happens -- so the fix
might be as simple as the attached patch?

Kenneth, are you able to try this out?

Cheers,

ellie

On Mon, Jul 25, 2016, at 02:02 PM, Bron Gondwana wrote:
> Thanks for reporting this.  Ellie, if you get a chance can you look at
> it?  I'm in the middle of the the FastMail security rollout right now, so
> I can't do anything today.
> 
> Bron.
> 
> On Mon, Jul 25, 2016, at 11:45, Kenneth Marshall wrote:
> > Hi,
> > 
> > I accidentally hijacked another thread. So here is a new message. I am
> > testing a Cyrus IMAP 2.3.16 to 2.5.9 upgrade and I am having problems
> > moving messages between folders before the 'reconstruct -V max' has been
> > run on the mailboxes. Here is the error from the IMAP client:
> > 
> > imap_copy_messages [a0080 NO Mailbox format corruption detected]?
> > 
> > and the details from the IMAP logs:
> > 
> > <1469384341<a0077 NOOP
> > >1469384341>a0077 OK Completed
> > <1469384354<a0078 LIST "" "%"
> > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0078 OK Completed (0.240 secs 356 calls)
> > <1469384361<a0079 STATUS "Drafts" (UIDVALIDITY)
> > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > a0079 OK Completed
> > <1469384361<a0080 UID COPY 1162729 "Drafts"
> > >1469384362>a0080 NO Mailbox format corruption detected
> > <1469384370<a0081 LIST "" "%"
> > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0081 OK Completed (0.230 secs 356 calls)
> > <1469384372<a0082 CLOSE
> > a0083 MYRIGHTS "Drafts"
> > a0084 SELECT "Drafts"
> > >1469384372>a0082 OK Completed
> > >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> > a0083 OK Completed
> > >1469384372>* 0 EXISTS
> > * 0 RECENT
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old \*)] Ok
> > * OK [UIDVALIDITY 1345619846] Ok
> > * OK [UIDNEXT 6] Ok
> > * OK [HIGHESTMODSEQ 1] Ok
> > * OK [URLMECH INTERNAL] Ok
> > * OK [ANNOTATIONS 65536] Ok
> > a0084 OK [READ-WRITE] Completed
> > >1469384355>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0078 OK Completed (0.240 secs 356 calls)
> > <1469384361<a0079 STATUS "Drafts" (UIDVALIDITY)
> > >1469384361>* STATUS Drafts (UIDVALIDITY 1345619846)
> > a0079 OK Completed
> > <1469384361<a0080 UID COPY 1162729 "Drafts"
> > >1469384362>a0080 NO Mailbox format corruption detected
> > <1469384370<a0081 LIST "" "%"
> > >1469384370>* LIST (\Noinferiors \HasNoChildren) "/" INBOX
> > * LIST (\HasChildren) "/" DSPAM_notspam
> > * LIST (\HasChildren) "/" DSPAM_spam
> > * LIST (\HasNoChildren) "/" Drafts
> > * LIST (\HasChildren) "/" Mail
> > * LIST (\HasNoChildren) "/" Sent
> > * LIST (\HasNoChildren) "/" Spam
> > * LIST (\HasNoChildren) "/" Trash
> > a0081 OK Completed (0.230 secs 356 calls)
> > <1469384372<a0082 CLOSE
> > a0083 MYRIGHTS "Drafts"
> > a0084 SELECT "Drafts"
> > >1469384372>a0082 OK Completed
> > >1469384372>* MYRIGHTS Drafts lrswipkxtecda
> > a0083 OK Completed
> > >1469384372>* 0 EXISTS
> > * 0 RECENT
> > * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old)
> > * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $mdnsent Old \*)] Ok
> > * OK [UIDVALIDITY 1345619846] Ok
> > * OK [UIDNEXT 6] Ok
> > * OK [HIGHESTMODSEQ 1] Ok
> > * OK [URLMECH INTERNAL] Ok
> > * OK [ANNOTATIONS 65536] Ok
> > a0084 OK [READ-WRITE] Completed
> > <1469384392<a0085 NOOP
> > >1469384392>a0085 OK Completed
> > <1469384412<a0086 NOOP
> > >1469384412>a0086 OK Completed
> > 
> > Is this a known issue with the upgrade process? Messages can be delivered to
> > the folders via a sieve script, but both message moves and copies fail. If there
> > is any fix for this issue, it would be greatly appreciated. I have been waiting
> > since the release of 2.4.x for a version that would handle the I/O storm caused
> > by automatically upgrading the mailbox format on first access, and 2.5.x does
> > manage that successfully. Thank you for any assistance.
> > 
> > Regards,
> > Ken
> > 
> > On Fri, Jul 22, 2016 at 02:27:01PM -0500, Kenneth Marshall wrote:
> > > On Sat, Jul 16, 2016 at 12:04:26AM +1000, Bron Gondwana via Info-cyrus wrote:
> > > > On Fri, Jul 15, 2016, at 23:52, Bron Gondwana via Info-cyrus wrote:
> > > > > On Fri, Jul 15, 2016, at 23:46, Andy Dorman via Info-cyrus wrote:
> > > > > > So if the issue apparently lies with twoskip, can we keep our dbs using 
> > > > > > skiplist and do the 2.4 -> 2.5 upgrade?  Is it possible -h could revert 
> > > > > > back to skiplist?
> > > > > 
> > > > > You can convert a db back just by changing the setting in imapd.conf and restarting - it will convert itself.
> > > > > 
> > > > > mboxlist_db: skiplist
> > > > > 
> > > > > > If it could help we could test upgrading to 2.5.8 on our dev server 
> > > > > > while leaving our database(s) as skiplist.
> > > > > 
> > > > > Twoskip should be fixed in 2.5.9.  I've worked out what went wrong, and am working on patches now :)
> > > > 
> > > > Patches have passed all the tests - they're on master and 2.5.  I'm pushing to FastMail's testing stores and then going to bed :)
> > > > 
> > > > -- 
> > > >   Bron Gondwana
> > > >   brong at fastmail.fm
> > > > ----
> > > 
> > > Hi Bron and Ellie,
> > > 
> > > Thank you for the new release. I am currently testing my upgrade process and while
> > > I can access my mailboxes after the upgrade to 2.5.x but before the 'reconstruct -V max',
> > > I cannot move messages between mailboxes. When I try I get the following error:
> > > 
> > > imap_copy_messages [a0012 NO Mailbox format corruption detected]?
> > > 
> > > and the move fails. Once I have updated the folder formats with the reconstruct,
> > > it works, but this causes a lot of issues with the upgrade. Have you seen this
> > > before and is there anything that can be done to make it work before the 
> > > reconstruct?
> > > 
> > > Regards,
> > > Ken
> > *** HEADER EXTRACTED deferred/C/CB16F4219066 ***
> > *** MESSAGE FILE END deferred/C/CB16F4219066 ***
> 
> 
> -- 
>   Bron Gondwana
>   brong at fastmail.fm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-imapd-2.5.9-record-cachecrc-check.patch
Type: text/x-diff
Size: 421 bytes
Desc: not available
URL: <http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20160725/c55de9cd/attachment-0001.bin>

More information about the Info-cyrus mailing list