Another cache bug!

[Bron Gondwana]

On Tue, 31 Mar 2009 15:51 -0700, "David R Bosso" <dbosso+lists.cyrus at lsit.ucsb.edu> wrote:
> --On December 10, 2008 10:36:56 AM +1100 Bron Gondwana
> <brong at fastmail.fm> 
> wrote:
> 
> > Wow, this is the thanks I get for doing sanity checks on files, find more
> > bugs!
> >
> > This one is due to delayed expunge, plain and simple.  Cyrus decides what
> > cache records to copy during an IMAP COPY command by reading the cache
> > offsets for msgno and msgno+1 (or the end of the cache file if it's the
> > last msgno).
> >
> > Obviously if some intervening messages have already been deleted from the
> > cyrus.index then it will be copying all those cache records as well to the
> > destination folder.  Oops.
> >
> > The attached patch reworks mailbox_cache_size into two functions, the
> > second being mailbox_cache_size_detail that takes memory locations for
> > the cache mmap rather than a mailbox object (because imapd doesn't update
> > the locations in the object correctly according to my testing, *sigh*.
> > Gotta love global variables)
> >
> > It then uses mailbox_cache_size_detail to calculate the ACTUAL record
> > length for this single cache item rather than blindly copying everything
> > up to the next index record's pointer.
> >
> > Also note: in the event of cache corruption, mailbox_cache_size_detail
> > returns zero bytes, which correctly makes append_copy re-parse the
> > message file.  It's all shiny :)
> >
> > Wes/Ken - please apply to CVS :)
> >
> > Thanks,
> >
> > Bron.
> > --
> >   Bron Gondwana
> >   brong at fastmail.fm
> 
> Going through my list of 2.3.13 patches to put together a local 2.3.14 
> build, I ran across this one.  It doesn't appear to have made it into 
> 2.3.14.  Oversight, or is there something wrong with it?

It's being reworked!  In particular, I'm working on delayed cache file loading.
There's also stuff about cache version number and locking all sort of floating
around...

For the immediate fix, I would recommend the attached patch.  It applies
directly on top of CVS.  Not so sure if it depends on anything that's not in
2.3.14... I don't _think_ so.

NOTE - this patch actually touches quite a lot of code.  This is the only bit
that's actually in production at FM at the moment - the delayed loading is
still being tested.

Still - this puts ALL cache accesses through a single, simple API :)

-- 
  Bron Gondwana
  brong at fastmail.fm

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cache.patch
Type: text/x-patch
Size: 50990 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20090401/c67c086b/attachment-0001.bin