"reconstruct -k" is still broken: do not use.

[David Carter]

Guess who thought that the quietest week of the year was an excellent 
chance to upgrade from 2.3.8 to 2.3.11 and "reconstruct -rkG" everything 
in sight to get shiny new SHA1 GUIDs? Fortunately I am sufficiently 
paranoid about my Cyrus machines that only a test system ended up with:

  Dec 21 22:57:07 cyrus-1 cyr_expire[23922]:
   IOERROR: reading cache record for user.dpc22:
   got bogus offset 577922368 for 1/2; try reconstruct

First up, a trivial xrealloc bug:

reconstruct_expunge() tracks no more than 300 expunged messages in a 
mailbox. Anything more than this becomes uninitialised space in a 
xrealloc()ed array: typically zeros on my platform. Consequently messages 
end up in both cyrus.index and cyrus.expunge when you reconstruct -k.

http://www-uxsup.csx.cam.ac.uk/~dpc22/cyrus/patches/2.3cvs/reconstruct3.patch

Rather more fundamental: "reconstruct -k" copies expunge.index records 
verbatim, without making any effort to update offsets into cyrus.cache. 
These offsets are used by both "cyr_expire -X" and "unexpunge".

cyrus.cache entries aren't constant: the cached headers and bodystructure 
information changes from version to version. Here is a before and after 
version just reconstructing a single cache file from 2.3.8 to 2.3.11

   -rw-r----- 1 cyrus cyrus 622680 2007-12-23 15:56 cyrus.cache-
   -rw-r----- 1 cyrus cyrus 626404 2007-12-23 15:56 cyrus.cache

I know how I plan to fix this problem: reconstruct needs to rebuild 
cyrus.index and expunge in parallel, rather like sync_combine_commit(). 
However this will be a rewrite of reconstruct.c, not a one line fix.

For the time being here is a small utility to clear GUIDs without the 
overhead (and hopefully without the brokenness) of "reconstruct -rgk":

http://www-uxsup.csx.cam.ac.uk/~dpc22/cyrus/patches/2.3cvs/reset_guids.patch

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.