lmtpd: deliver.db checkpointing

[Patrick Boutilier]

On 11/15/2010 10:36 AM, David Mayo wrote:
>
> One day last week I noticed a performance dip on our IMAP server. We
> tracked the cause down to periodic "checkpointing" of the deliveries
> database by the LMTP daemon on our backend server.
>
> The only real effect on users is that deliveries don't happen for the
> duration of the checkpointing cycle, although on this occasion I noticed
> the IMAP server was running unusually slow during this period and I also
> discovered the MUPDATE server was unavailable shortly afterwards for
> creating new mailboxes. This is presumably because the MUPDATE server
> was busy in the backed-up delivery process.
>
> Having looked at the source code, we can see that ctl_cyrusdb does not
> touch deliver.db and it is not possible to force checkpoints on or off
> within lmtpd. Ideally we want to schedule the checkpointing to a "less
> sociable" time.
>
> On our setup, the checkpoint runs every couple of days and usually takes
> 4-6 minutes[1]. We run cyr_expire every night to remove entries older
> than 3 days from the duplicate deliveries database but this is the only
> maintenance we can schedule on this database.


No solution, but I wonder why your checkpoint takes so long? With 
similar DB size it take only seconds here.



Nov  8 21:18:04 student2 lmtpunix[6541]: skiplist: checkpointed 
/var/imap/deliver.db (650493 records, 67180160 bytes) in 5 seconds

Nov 12 23:14:06 student2 lmtpunix[26425]: skiplist: checkpointed 
/var/imap/deliver.db (768262 records, 80010936 bytes) in 11 seconds





>
> Do other people suffer from this? Is there a patch that anyone has
> written and/or will be submitting upstream any time soon?
>
> We are running Cyrus 2.3.13 across a front end, a back end and a
> replication host for around 25,000 users.
>
> Regards,
>
>
> Dave.
>
> David Mayo
> Networks/Systems Administrator
> University of Bath Computing Services, UK
>
> [1]
> Nov  4 15:32:33 imap.bath.ac.uk lmtp[29595]: [ID 301543 mail.info]
> skiplist: checkpointed /opt/etc/imapd/deliver.db (787618 records,
> 74775536 bytes) in 386 seconds
> Nov  9 14:26:06 imap.bath.ac.uk lmtp[1676]: [ID 301543 mail.info]
> skiplist: checkpointed /opt/etc/imapd/deliver.db (386167 records,
> 36638408 bytes) in 232 seconds
> Nov 11 12:45:35 imap.bath.ac.uk lmtp[2497]: [ID 301543 mail.info]
> skiplist: checkpointed /opt/etc/imapd/deliver.db (657254 records,
> 62540568 bytes) in 396 seconds
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: boutilpj.vcf
Type: text/x-vcard
Size: 286 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20101115/7715f172/attachment.vcf