competition

[Bron Gondwana]

On Wed, 22 Sep 2010 09:12 +0200, "Sebastian Hagedorn" <Hagedorn at uni-koeln.de> wrote:
> --On 22. September 2010 16:10:15 +1000 Bron Gondwana <brong at fastmail.fm> 
> wrote:
> 
> > Now - BDB database SHOULD be upgradable.  I want to find a BDB expert
> > to help me with that - (a) detecting that an upgrade is necessary, and
> > (b) doing the upgrade.
> 
> I wouldn't exactly call myself an expert, but I think I understand the 
> basics of BDB operation. The biggest problem is that there may be
> multiple 
> version of the BDB utilities installed. So you need to know which BDB 
> version Cyrus was linked against and somehow figure out what the 
> corresponding tools are on the box you're on. The tool are named db_stat, 
> db_verify, db_upgrade etc., but it's not unusual to have db41_stat, 
> db42_stat etc .
> The "file" utility is pretty useful, because it tells you the version of
> a 
> BDB file. Unfortunately that version has nothing to to with the version 
> number of the BDB libraries themselves. Instead they look like this:
> 
> Berkeley DB (Hash, version 8, native byte-order)

Yeah, I can get that just by reading the value as byte 12 in the file.  I pulled a copy of the BDB source and grabbed their magic numbers already.

The thing is - I don't want to be calling out to the tools - I want to be doing it in_process with a Cyrus tool that's already compiled against the correct version of BDB.

> I suppose one could do this:
> - at startup, create a new BDB hash file using a Cyrus binary
> - check its version number using "file"
> - compare that to the existing BDB files
> - if they're different you need to perform an upgrade

Oooh.  Actually, that's a point.  Create a temporary file of each type and pull the version field from it, then check each BDB file's version field.  Hmm. That's quite viable actually.  I like it :)

> The upgrade itself *should* be doable using db_upgrade (using the version 
> that goes with the libraries Cyrus is linked against), although I think 
> there are cases where you have to use db_dump followed by db_load. The 
> problem with the latter is that you need the dump from the *old* version
> of 
> the libraries!

http://download.oracle.com/docs/cd/E17076_02/html/upgrading/index.html seems to have good information on "what's changed" in each version.

On the topic of upgrades:
http://download.oracle.com/docs/cd/E17076_02/html/programmer_reference/am_upgrade.html

"We do guarantee that any archived database may be upgraded using a current Berkeley DB software release and the DB->upgrade() method, and there is no need to step-wise upgrade the database using intermediate releases of Berkeley DB. Sites should consider archiving appropriate copies of their application or application sources if they may need to access archived databases without first upgrading them."

> > .... given the issues with BDB.  Is it worth embedding a copy of
> > BDB into the Cyrus distribution rather than using the OS one?  I
> > know it's generally considered bad taste, but it sure makes
> > keeping in sync easier!
> 
> That would certainly help with the issues I described above, but only if 
> you either also include the utilities or have your own binaries that 
> fulfill their function.

No need I don't think if we do the little trick above to figure out BDB version and upgrade.  Ideally, I'd like to not be using BDB because the incompatibilities are a killer.  If we could just say "use a backwards compatible format" to BDB and have it readable by any 3.x + version, I'd be a very happy camper.

Bron.
-- 
  Bron Gondwana
  brong at fastmail.fm