Berkeley DB upgrade - 2nd try

[Paul Boven]

Hi everyone,

Another follow-up to my monologue thread - I've had a few helpful 
replies off-list though, and comments (on or off list) are very much 
appreciated.

> I've started building cyrus-imapd-2.2.13 - the Cyrus homepage still 
> lists 2.2.12 as the 'current' version, but is apparently outdated a bit. 
> In another posting to this list, Andrew Morgan hinted that "Cyrus 
> v2.2.12 and older will not work with Berkeley DB 4.3+ without a patch". 
> With 2.2.13, at least I get a meaningful error out of Berkeley when I 
> start Cyrus:
> 
> DBERROR db4: Skipping log file /var/imap/db/log.0000000001: historic log 
> version 7
> DBERROR db4: /var/imap/db/log.0000000002: log file open failed: No such 
> file or directory
> DBERROR db4: PANIC: No such file or directory
> 
> Running db_recover from Berkeley4.4.20 gave me essentially the same errors.
> 
> In the end, I figured it apparently doesn't care about log.00000001, so 
> I moved it aside. This time, it also stopped looking for log.00000002 
> and recovered succesfully. And now Cyrus works again!
> 
> Before I even want to consider doing this kind of surgery on any of my 
> production Cyrus servers, I'd like to know what would have been in 
> log.00000001 - what risks do I run when removing it?

The above procedure is not the correct way to go about, I've found. 
Cyrus does work in the sense that I can open my existing mail 
(mailboxes.db is skiplist), but sending a new message into the system 
caused LMTP to complain about the state of the deliver.db, and one cpu 
becomes fully loaded with kernel tasks, probably lots of syscalls.

So what would the correct procedure be? I'm upgrading from Berkely 
4.1.25 to 4.4.20. I've looked at the Berkeley documentation, and found 
this on upgrading:

 > Otherwise, if the application has a Berkeley DB transactional
 > environment and the databases need upgrading, the application may
 > be installed in the field using the following steps:

Given that there are 'log' files in the DB environment, I'm assuming 
this is indeed a transactional Berkeley DB. Because the database format 
changed between 4.1.25 and 4.4.20, the above case applies. The procedure 
then would be:

1. Shut down the old version of the application.
2 Run recovery on the database environment using the DB_ENV->open method 
or the db_recover utility.
3. Remove any Berkeley DB environment using the DB_ENV->remove method or 
an appropriate system utility.
4. Archive the database environment for catastrophic recovery. See 
Archival procedures for more information.
5. Recompile and install the new version of the application.
6. Upgrade the application's databases. See Upgrading databases for more 
information.
7. Archive the database for catastrophic recovery again (using different 
media than before, of course). Note: This archival is not strictly 
necessary. However, if you have to perform catastrophic recovery after 
restarting the application, that recovery must be done based on the last 
archive you have made. If you make this second archive, you can use it 
as the basis of that catastrophic recovery. If you do not make this 
second archive, you have to use the archive you made in step 4 as the 
basis of your recovery, and you have to do a full upgrade on it before 
you can apply log files created after the upgrade to it.
8. Force a checkpoint using the DB_ENV->txn_checkpoint method or the 
db_checkpoint utility.
9. Restart the application.

(source: http://www.sleepycat.com/docs/ref/upgrade/process.html)

Quite an involved procedure. Some people have suggested dropping 
Berkeley alltogether, but I feel it's worth the effort to actually gain 
some understanding of this complex environment, and for the added 
performance. So next order of business is to downgrade my test 
environment, then re-upgrade it and test the procedure above.

Question: 'Remove any Berkeley DB environment using the DB_ENV->remove 
method or an appropriate system utility'. I guess 'rm -rf 
/var/imap/db/log* /var/imap/db/_*' would be 'appropriate'? (this way, I 
don't delete the skipstamp file).

Regards, Paul Boven.