More Skiplist Stuff!

[Bron Gondwana]

On Mon, Dec 31, 2007 at 02:10:49PM +1100, Bron Gondwana wrote:
> On Sun, Dec 30, 2007 at 11:38:36PM +1100, Bron Gondwana wrote:
> > I have much bigger fish to fry here, mailboxes.db corruptions again
> > indeed.  I've seen a couple of them over the past few days, with
> > behaviour that looks remarkably like it might be a Linux kernel bug
> > with mmap, seek and file ends.  Arrgh.
> 
> Ho hum - there's also this little bit of trouble with the 
> SAFE_TO_APPEND code that I discovered reading the code again.
> I don't know if this can explain everything, but it is still
> worth fixing.
> 
> Basically, if you hand a pointer to a null transaction to one
> of the read functions (myfetch, myforeach) then it locks the
> file in write mode and begins a new transaction... but it
> doesn't check SAFE_TO_APPEND.
> 
> Later you do a write, and it still doesn't check SAFE_TO_APPEND
> because it already has a transaction.
> 
> This patch fixes the problem by moving the SAFE_TO_APPEND check
> into newtxn so it gets called from all codepaths that start a
> new transaction.

Patch the third (and renewing a fourth!)

The third patch is lock and open-status tracking.

The renewed one is:
http://cyrus.brong.fastmail.fm/patches/cyrus-skiplist-transactions-2.3.10.diff

This didn't apply when I was building 2.3.11, and stupid me went
"ahh, Ken must have accepted it since it's really quite important
to have something like this".  Gosh.  No wonder our mailboxes.db
reliability has been dropping off again.

The new one is what found this, and is the _real_ solution to lock
safety - code that asserts that the locking status of the file is
what we expect it to be by tracking both the lock status and an
is_open allowing us to re-read the entire header safely in each
locking function.

http://cyrus.brong.fastmail.fm/patches/cyrus-skiplist-state-2.3.11.diff

I haven't put a description on this one, but it's reasonably clear
from the code I hope.

Bron ( going to get some rest now! )
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-skiplist-transactions-2.3.10.diff
Type: text/x-diff
Size: 10315 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080102/9669d3ff/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-skiplist-state-2.3.11.diff
Type: text/x-diff
Size: 6827 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080102/9669d3ff/attachment-0003.bin