Sophia database => fail

[Bron Gondwana]

On Mon, Jun 20, 2016, at 18:36, Howard Chu via Cyrus-devel wrote:
> Thomas Jarosch via Cyrus-devel wrote:
> > plain old Berkeley DB 6.x might be an option, too.
> > It has transaction support, page level locking and is a key-value store.

We've had bad experiences with BDB in the past - there's a reason why it got pulled from Cyrus.  The upgrade story was a mess - BDB upgrade headaches have been the major cause of people staying with old versions.

> > Not sure about the licensing issues though:
> > http://www.infoworld.com/article/2611450/open-source-software/oracle-switches-berkeley-db-license.html
> > http://www.oracle.com/technetwork/products/berkeleydb/downloads/licensing-098979.html
> 
> Licensing is one of the reasons we have abandoned BerkeleyDB in OpenLDAP. 

That too.  Nobody wants to use a database owned by a lawnmower:

https://www.youtube.com/watch?v=-zRN7XLCRhc&t=34m

> That, and LMDB is orders of magnitude faster than BerkeleyDB.
> > Also it's not a "multi-level" database.

Well, yeah.  That's not totally necessary.  Btree and LSM database structures both provide what we need in terms of operations.

I'm going to try to build an LMDB backend.  I do have some questions for Howard about how best to do it... :)  The CyrusDB abstraction layer allows opening multiple databases by filename.  Should we be opening one environment per file, or just a single environment per process?  Looking at:

http://lmdb.tech/doc/starting.html

We could us MDB_NOSUBDIR to use the file path as a data file via mdb_env_open...

We already use internal "open file" tracking in most of our database types, so I'd add that for the LMDB backend as well.  I just use a linked list, since we tend to have at most 3-4 databases open at once in each process.

We don't use threading for anything except mupdate, and that uses a mutex around all the database access, so multiple thread synchronisation should be OK :)

Bron.

-- 
  Bron Gondwana
  brong at fastmail.fm