possible seen flag db corruption

[Bron Gondwana]

On Thu, Dec 12, 2013, at 08:29 AM, David Mansfield wrote:
> On 12/11/2013 03:49 PM, Bron Gondwana wrote:
> > On Thu, Dec 12, 2013, at 05:33 AM, David Mansfield wrote:
> >> Hi All,
> >>
> >> I'm tracing a problem that is reproducible on cyrus-imapd 2.3.16 on
> >> centos 6, that occurs when messages are delivered to a mailbox
> >> while a client is setting flags and retrieving existing messages.
> >
> > This sort of thing is a problem with 2.3.x and earlier
> > unfortunately.
> >
> > I spent a couple of years rewriting all the internals for 2.4 so
> > that changes really are atomic and seen state is managed within the
> > same transaction as everything else.
> >
> > Bron.
> >
>
> Thanks for the response, Bron. I figured as much - I'm imagining some
> un-initialized bitfield somewhere...
>
> Since you mentioned it ("same transaction as everything else"), I have
> a quick question about EXPUNGE.  If one session has done a SEARCH (say
> on message id), and another session does expunge and some message is
> removed at a lesser sequence number, then the first session does STORE
> to flag that message based on the sequence number returned by the
> SEARCH (say \Deleted), is that protected by a transaction or is there
> a race condition in this scenario.

In 2.4 and above the behaviour is exactly per the standard, which is...

each TCP connection gets its own sequence numbers.  They have absolutely
no relationship to the sequence numbers that other connections have.

So the first session will attempt to store to an already expunged
record. Here's what happens:


    for (msgno = 1; msgno <= state->exists; msgno++) {
        im = &state->map[msgno-1]; 
        if (!seqset_ismember(seq, storeargs->usinguid ? im->uid : msgno))
            continue;

        /* if it's expunged already, skip it now */
        if ((im->system_flags & FLAG_EXPUNGED))
            continue;

The loop in index_store checks the flags, and discovers that it's an
already expunged message.  It just skips any action on the message.  The
store will "succeed", because it doesn't set any error - though if it
wasn't a FLAGS.SILENT you will be able to tell because there's no
updated response.

So it's perfectly safe, so long as you're listening for * EXPUNGE N
responses to any IMAP command which is allowed to give them during
session 1, because they do - of course - change the sequence numbers
of the messages above them.

Cheers,

Bron.

-- 
  Bron Gondwana
  brong at fastmail.fm