What's happening in future branch - April 19

[Bron Gondwana]

On Thu, Mar 25, 2010 at 10:28:53PM +1100, Bron Gondwana wrote:
> index.c - use a mailbox object
> ==============================
> 
> All the globals go away!  There's just a global mailbox struct, which
> will have a lock_fd in shared mode, meaning that all the offsets into
> the cyrus.index and cyrus.cache mmaps are ALWAYS VALID.
> 
> Along with compulsary CONDSTORE - vast swathes of code complexity are
> being removed :)  You don't need to track times when you have modseq
> to determine if flags should be sent again.  You don't have to keep
> statting files, just re-read the header and then scan for updates.
> 
> It WILL be necessary to keep a global array mapping from msgno to recno
> so non-UID commands can be efficient.  This can be rewritten whenever
> highestmodseq is detected to have changed :)

Not only have the globals in mailbox.c gone away, but I bit the bullet
and rewrote index.c to use a "struct index_state" rather than globals
as well - and then rewrote all the functions to take an index_state
pointer rather than a mailbox pointer.  This means that you can have
multiple index states open at once - which makes things like url_catenate
not suck quite so hard!  Before they had to throw away all the mapping
and re-create it...

Looking around the rest of the code - I think pop and nntp are all good.
lmtp seems fine, though it's still untested at the moment.  I have a
bunch of edge-case checking to do, but most of the everyday functions
seem fine.

I have broken non-UID commands slightly, which will just required figuring
out which instances of index_check need to get the "don't screw with
EXPUNGES" flag set correctly, and that should be good.

The big TODO list contains:

* check QRESYNC behaviour, particularly with respect to ranges
* test cyr_expire-esqe functionality.  Possibly split the cyr_expire tool
  up in to 3 different tools that do one thing well rather than one which
  does 3 things poorly.
* replication - take advantage of the new fields.

There's still some thought required about cache_offset.  I'm in two minds
about requiring the cache_offset to be the same at both ends, but the
CRCs won't match up otherwise unless I add a separate "just the important
fields" checksum!

There might be some benefit to doing that actually - because otherwise
upgrades will kind of suck.  It would be good to have replication work
reliably across point upgrades in to the future, but adding a field to
the cached headers would break it.

Anyway - the code is going to look pretty unfamiliar to everyone!  Lots
of changes.  Diffstat says over 6000 lines plus and over 8500 lines
negative!  Still got a week and a bit to make it stable :)

Bron.