Automatic database coversion script anyone?

[Ondřej Surý]

On Tue, Apr 5, 2011 at 01:08, Bron Gondwana <brong at fastmail.fm> wrote:
> On Tue, Apr 05, 2011 at 12:28:02AM +0200, Ondřej Surý wrote:
>> > I really, really wanted to have BDB able to handle in-place upgrades
>> > as well, but I haven't been able to figure out a way.  There is an
>> > ->upgrade method on databases, but it says it's destructive and can
>> > fail - so you can't just run it during the startup process and hope...
>> > unless you take a backup first!
>> >
>> > Eventually my goal is that you can change the DB type in imapd.conf
>> > and restart Cyrus, and it will just work.  We're nearly there - it
>> > will automatically convert between types (by reading the magic from
>> > the top of the file) - but it just can't upgrade BDB.  Answer that
>> > question, and I can give you a 2.4.x release that upgrades magically!
>>
>> What if you add an option to cyrus-imapd (to all BDB users) which
>> would say it can do the in place upgrade. The DB_ENV->upgrade looks
>> like it is OK, the only problem is that you need to have backups - and
>> that something which can be done either by maintainer script in
>> postinst or by administrator - it would also help if there was a
>> program which could list all the databases which needs to be
>> converted/backuped.
>>
>> If there was a such option I could easily accomplish some magick with
>> depending on dbX.Y-util (where X.Y is the old version of BDB) and
>> backup the old database and only after the backup is done (or the
>> users say "no" to a backup question because he has the backups already
>> or he doesn't care) I could run the cyrus-imapd --do-upgrade (or
>> cyrmaster --do-upgrade). That would satisfy both - the safety of the
>> databases and the need for seamless upgrade.
>>
>> How does that sounds?
>
> Fantastic :)  Have a look at the "magic" inside ctl_cyrusdb.  One
> option (since we default to all non-bdb now) would be to always take
> a backup first if it's a BDB file.  Then try the upgrade of the backup
> file, and if THAT works, then create a new skiplist or BDB file in the
> target place.  Make startup a bit slow if you insist on using BDB of
> course.

You cannot backup (using DB_ENV->archive) before you upgrade, but if
you find a way how to call (backend)->init with CYRUSDB_UPGRADE flag
and them (backend)->open with same flag, the attached patch should do
the trick. (But I advise to review the patch - it's 3AM here...)

O.
-- 
Ondřej Surý <ondrej at sury.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CYRUSDB_UPGRADE.patch
Type: application/octet-stream
Size: 2166 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20110405/ce7f2361/attachment.obj