ctl_cyrusdb questions, looking for answers
brong at fastmail.fm
Tue Mar 17 19:26:06 EDT 2009
On Tue, Mar 17, 2009 at 09:37:37AM -0400, Adam Tauno Williams wrote:
> > > Not exactly sure what you mean here. I think you only need the database in
> > > db.backup? and of course clean the environment before restoring such a db.
> > AFAIK only that three databases are copied to the db.backupX directory. So if
> > there are other checkpointed dbs they stay in the original place and I need to
> > backup them?
> As I understand it: there is no reason to backup deliver.db,
> statuscache.db, or tls_sessions.db. You can dump those without
> consequence and they will be rebuilt on the fly.
Yes - we delete these as part of our startup scripts (prefer not to
have any BDB files lying around after a restart)
Hey "checkpoint" is a massively overloaded term. In skiplist,
a "checkpoint" is when the file gets rewritten in order. It's
a complete rewrite of the file. Otherwise, skiplist is append
only, so deleted or replaced records take up space.
The algorithm for checkpoints is currently:
tid->logend > (2 * db->logstart + SKIPLIST_MINREWRITE)
Which basically means checkpoint when the file is over half
garbage. I think that's actually a little too rare, especially
with SKIPLIST_MINREWRITE = 16834, it means .seen files have a
lot of junk in them!
The "checkpoint" that ctl_cyrusdb does on the other hand, is
totally different. It's just copying those files into the
backup directories. It doesn't actually run a db checkpoint,
at least not on skiplist. There's no way to trigger one
I have patches in the FastMail set that tune SKIPLIST_MINREWRITE
lower, and the amount of junk required lower. They also do a
checkpoint every time we run recovery (recovery is basically
going through and recreating all the links between records,
rolling back any uncompleted transaction as well).
I think the "checkpoint on recovery" code will go into 2.3.15 -
it makes sense to rewrite the file when it's hot.
More information about the Info-cyrus