expunge_days option in Cyrus 2.4

[David Carter]

We moved from Cyrus 2.3 to 2.4 earlier in the year. We run with delayed 
expunge, with a nightly job to expire old expunged messages after 28 days:

   cyr_expire -E 3 -D 28 -X 28

This worked nicely under Cyrus 2.3.

A few days back we had our first request to restore email which has been 
expunged more than seven days previously, and I made the unpleasant 
discovery that we were only actually keeping expunged messages for 7 days.

A bit of digging through source code uncovered:

   expunge_days: 7
     Number of days to retain expunged messages before cleaning up
     their index  records. The  default is 7. This is a necessary for
     QRESYNC to work  correctly. If combined with delayed expunge (above)
     you will also be able to unexpunge messages during this time.

which seems to expire messages as soon as a mailbox is closed after 7 
days, regardless of what cyr_expire might be up to. I believe increasing 
expunge_days to 30 days will make things work the way that I expected.

I put this down to a bad experience and failing to trawl through 
imap.conf.5 carefully enough looking for new options (it isn't mentioned 
in changes.html, but a lot changed between 2.3 and 2.4). I have since 
found a short thread on this list from January discussing the new option.

However I had a second bad experience with expunge_days over the weekend.

We use the replication engine to seed new mailstores and move users around 
the cluster. This worked nicely using my own code in Cyrus 2.1 and 2.3.
In 2.4 the following innocent looking test:

sync_client.c :

     /* check that replication stands a chance of succeeding */
     if (remote && !is_repeat) {
 	if (mailbox->i.deletedmodseq > remote->highestmodseq) {
 	    syslog(LOG_NOTICE, "inefficient replication ("
 		   MODSEQ_FMT " > " MODSEQ_FMT ") %s",
 		   mailbox->i.deletedmodseq, remote->highestmodseq,
 		   local->name);
 	    r = IMAP_AGAIN;
 	    goto done;
 	}

appears to kick in if you run "sync_client -u" between a master and 
replica or backup mailstore with a gap of more than 7 days between the two 
runs. mailbox_full_update() gets rather excited about messages on the 
replica which have been deleted, expunged and unlinked from the master.

Again "expunge_days: 30" should help, but I thought that others might want 
to be warned before they try to use sync_client to move users around.

-- 
David Carter                             Email: David.Carter at ucs.cam.ac.uk
University Computing Service,            Phone: (01223) 334502
New Museums Site, Pembroke Street,       Fax:   (01223) 334679
Cambridge UK. CB2 3QH.