cvt_cyrusdb and berkeley log files

[Ken Murchison]

Andreas wrote:

> While playing with the cvt_cyrusdb utility (and Simon's excellent
> cvt_cyrusdb_all script) I came across this problem when converting
> a database *from* berkeley *to* skiplist.
> 
> Let's say I have /var/lib/imap/mailboxes.db in berkeley format. I also
> have berkeley transaction log files in /var/lib/imap/log.* and
> /var/lib/imap/db/log.*.
> 
> I then shut down the cyrus daemon and convert mailboxes.db to skiplist.
> The conversion succeeds:
> # file mailboxes.db
> mailboxes.db: Cyrus skiplist DB
> 
> Now I restart cyrus. And get this in the logs:
> ctl_cyrusdb[30046]: DBERROR db4: /var/lib/imap/mailboxes.db: unexpected file type or format
> ctl_cyrusdb[30046]: DBERROR db4: Recovery function for LSN 1 4471396 failed
> ctl_cyrusdb[30046]: DBERROR db4: PANIC: Invalid argument
> ctl_cyrusdb[30046]: DBERROR: critical database situation
> 
> It seems ctl_cyrusdb is attempting to read mailboxes.db as a DB file and not skiplist.
> I *suppose* this happens because it is trying to recover the database using the log file
> which was left there and contains BDB transactions. AFAIK, however, this type of recovery
> procedure is called "catastrophic" by the berkeley folks: a normal recovery procedure
> would only touch log files which contain uncommited transactions. Looking at the code, I
> see that it is not performing catastrophic recovery (it's just using the DB_RECOVER flag
> when opening the environment), so I'm at a loss here.
> 
> The only way to remedy this situation is to remove the relevant log files.
> Anybody care to comment?

That's what I have done in the past.  The problem is that the log files 
still have references to the now nonexistent mailboxes.db.  What needs 
to happen is that the mailbox in question needs to be removed from the 
BDB environment so we don't try to recover it.  Its possible that doing 
a checkpoint (ctl_cyrusdb -c) before the conversion may solve the 
problem, but I have never tried it.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html