Sophia database => fail

[Howard Chu]

Bron Gondwana wrote:
> On Sun, Jun 19, 2016, at 07:49, Howard Chu wrote:
>> Bron Gondwana via Cyrus-devel wrote:
>>> So I wrote a backend for this thing:
>>>
>>> http://sophia.systems/
>>
>> Sorry you wasted your time. Even if you got that thing to "behave" as you
>> intended, it would have crashed on you anyway.
>>
>> https://github.com/pmwkaa/sophia/issues/54
>> https://github.com/pmwkaa/sophia/issues/53
>>
>> Actually, just look at all of the open issues on that repo. These folks have
>> No Clue what they are doing.
>>
>> It's a sick joke, not a usable database library.
>
> Hah, right.  It's hard to tell because their coding style is so minimalist - short variable names, no comments.  Good to know not to spend any more time on it.
>
>
>>     -- Howard Chu
>>     CTO, Symas Corp.           http://www.symas.com
>
> So you'll be happy to know that lmdb is on my list of things to integrate and try as well :)
>
> I've also been looking at Rocks DB. Unfortunately their C wrapper (it's
> C++)
doesn't seem to include the transaction support, so I'll have to either extend
it or write my own.
>
> One major goal here, and the reason I'm looking at multi-level databases
> in
particular, is the ability to have most of the unchanging data uploaded to an
object store, and only need to push a small log file each time something changes.

If by multi-level you mean LSM style DBs, you should be aware that these are 
extremely fragile in the face of application and OS crashes. It's a 
consequence of the fact that the DB state and metadata is scattered across 
multiple files, and you cannot atomically update all of them at once. LevelDB 
and RocksDB are both prone to self-corruption after a crash.

(Google "rocksdb corruption" ...)

Not quite sure I understand your use case. It sounds like you'll maintain a 
local store, with recent records, and a remote store with older records. Is 
that correct?

One of the features we'll be adding in LMDB 1.0 is incremental backup. This 
will write out all the DB pages that are newer than a particular 
transactionID. (A prototype of this feature is in the "incre" branch 
https://github.com/LMDB/lmdb/tree/incre but it needs some rework before 
releasing in 1.0)

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/