lmtpd: deliver.db checkpointing

[David Mayo]

One day last week I noticed a performance dip on our IMAP server. We 
tracked the cause down to periodic "checkpointing" of the deliveries 
database by the LMTP daemon on our backend server.

The only real effect on users is that deliveries don't happen for the 
duration of the checkpointing cycle, although on this occasion I noticed 
the IMAP server was running unusually slow during this period and I also 
discovered the MUPDATE server was unavailable shortly afterwards for 
creating new mailboxes. This is presumably because the MUPDATE server 
was busy in the backed-up delivery process.

Having looked at the source code, we can see that ctl_cyrusdb does not 
touch deliver.db and it is not possible to force checkpoints on or off 
within lmtpd. Ideally we want to schedule the checkpointing to a "less 
sociable" time.

On our setup, the checkpoint runs every couple of days and usually takes 
4-6 minutes[1]. We run cyr_expire every night to remove entries older 
than 3 days from the duplicate deliveries database but this is the only 
maintenance we can schedule on this database.

Do other people suffer from this? Is there a patch that anyone has 
written and/or will be submitting upstream any time soon?

We are running Cyrus 2.3.13 across a front end, a back end and a 
replication host for around 25,000 users.

Regards,


Dave.

David Mayo
Networks/Systems Administrator
University of Bath Computing Services, UK

[1]
Nov  4 15:32:33 imap.bath.ac.uk lmtp[29595]: [ID 301543 mail.info] 
skiplist: checkpointed /opt/etc/imapd/deliver.db (787618 records, 
74775536 bytes) in 386 seconds
Nov  9 14:26:06 imap.bath.ac.uk lmtp[1676]: [ID 301543 mail.info] 
skiplist: checkpointed /opt/etc/imapd/deliver.db (386167 records, 
36638408 bytes) in 232 seconds
Nov 11 12:45:35 imap.bath.ac.uk lmtp[2497]: [ID 301543 mail.info] 
skiplist: checkpointed /opt/etc/imapd/deliver.db (657254 records, 
62540568 bytes) in 396 seconds