Berkeley DB: private copy?

[Simon Matter]

> On Wed, Sep 22, 2010 at 04:43:23PM +0530, Shuvam Misra wrote:
>> I'm open to criticism, but I strongly feel Cyrus should carry its own
>> version of some of these critical system libraries, specially those ones
>> which have caused so much compatibility grief in its history. I know
>> this
>> is considered 'bad taste', but the sysadmin who just wants a stable
>> Cyrus
>> server probably cares less about taste than stability.
>
> It's quite justifiable in the case of BDB.  Unfortunately, not so much in
> the case of SASL which is the other usual suspect.  SASL tends to be
> deeply integrated into the rest of the system.
>
> The other interesting one is openssl - but I don't think we have many
> problems with it.  Not having openssl kind of sucks with 2.4 though.
> Would suck less if we put our own crc32 and sha1 implementations in
> the code.
>
>> BTW, a basic question: why do we face more problems with Berkeley DB
>> than
>> with other file formats?
>
> Environments mainly.  It does this funky "environment" thing which makes
> it very susceptible to any errors in open/close counts between processes.
> It's really not as robust in a multi-processor shared environment as
> one could hope.

The other big problem is the different on disk format of the different BDB
version and that not all newer BDB versions can upgrade any older file.

>
> And we're probably using it wrong.  Except I don't know how, or what
> to change.  Skiplist is nicely self-contained and relatively bug-free
> these days.  We use pure skiplist at FastMail, and we still get
> scads of DBERROR messages when we start and stop due to BDB environment
> counts going negative.

We do also skiplist only for years now - and everybody using our RPMs in
default configuration does so - and I'm very pleased how it works. That's
why I think it should be possible to build Cyrus completely without BDB
support. It could be an optional compile time option.

Simon