Sophia database => fail

Howard Chu hyc at
Sat Jun 18 18:59:42 EDT 2016

Howard Chu via Cyrus-devel wrote:
> Bron Gondwana wrote:
>> A good example of what I want is the way that the xapianactive file works in
>> Cyrus
>> search at FastMail:
>> Because only the most recent database is writable (in this case on tmpfs,
>> because
>> we don't need 100% reliability for search, it only takes about 20 minutes to
>> scan
>> every mailbox and reindex the stuff that was on tmpfs after a crash)

Also, since you're using tmpfs, this in-memory benchmark is relevant.
>> Every other database is read-only - and you can compact multiple of them
>> together
>> into a single database and then atomically switch the old ones out and the
>> new one
>> in with a single very quick xapianactive rewrite - so it's acceptable to
>> stop the world
>> while doing that.
> This sounds like a lot of bother, particularly the bit about "checking if
> tmpfs is full". It's also a bit confusing because you talk about "compacting"
> which I interpret as "cleaning out empty/unused space inside a DB" but in
> context it sounds like you really mean "merging" - combining multiple DBs into
> a single DB.
> If I were building this system with LMDB there would be no separate temp and
> meta tiers. LMDB would just mmap the DB on the SSD and let the OS buffer cache
> keep the hot pages in RAM. I'm not really sure I'd bother with multiple DBs
> either, there's nothing to compact. The data tier would be no different from
> the meta tier.
> When you say you can reindex the stuff on tmpfs quickly, that means you're
> only reindexing the most recent N emails?

   -- Howard Chu
   CTO, Symas Corp. 
   Director, Highland Sun
   Chief Architect, OpenLDAP

More information about the Cyrus-devel mailing list