competition

[Bron Gondwana]

On Mon, Sep 27, 2010 at 09:38:08AM -0400, Jeff Eaton wrote:
> 
> > Better to just use an internal DB codebase (like skiplists) that has
> > nothing to do with Sleepycat.  But then someone has to write and maintain
> > this code.
> >
> > I think the best compromise I've heard yet is to use something like
> > skiplists by default and make the use of libdb an optional feature like the
> > use of mysql.
> 
> I'd be in favor of "deprecating" the use of BerkeleyDB in Cyrus.  Making
> at least the default for new installations to be all-skiplist would be a
> good step in that direction, as well as encouraging people to move away
> from BDB where possible.

Just committed this change to dev/2.4.0-alpha1.  Along with the code
that automagically converts databases on ctl_cyrusdb -r, this should
make most sites that are running the default layout transparently
start running all-skiplist without even realising.
 
> Ideally, I'd suggest making the default behavior of "./configure; make;
> make install" to _not_ link in BDB at all unless --with-bdb is explicitly
> given.  As I recall, you have to jump through a few hoops to get a build
> of Cyrus to not link in the BDB libraries, even if you're not using them.

I haven't tried this.  Maybe I should.

> All of that said, it's probably worth directing future Cyrus DB backend
> work away from BDB and more to improving SQLite or other integration.

What about Hamster, or any of the other similar things?  What about
memcached for tls, pts and statuscache.  Worth looking at.

By the way - I'm travelling for the next few weeks, including meeting Ken
and Dave in Buffalo mid next month.  We have some new programmers starting
with FastMail/Opera in Melbourne while I'm away, and I'm hoping to dedicate
a large part of one of them's time to improving Cyrus.  My shortlist of
tasks to look at while I'm away is:

1) figure out what's up with BDB
2) seen db cleaner.  Remove stale entries from seen databases (including
   entries for user's own mailboxes now that the owner's seen is absorbed
   back into the cyrus.index)

Apart from these two things, I suspect most of the work we'll do on Cyrus
over the next few months will be quite "tactical" in nature.  Targetted
towards making cross-datacentre replication better, and working towards
supporting "conversations" (cross folder threads) in an efficient way.

And there'll be a fair bit of work on the supporting code that manages
replication from the outside too.  Possibly pulling some back into a C
daemon within Cyrus that can track replication status and monitor how
the replication is running.  Particularly of interest here is safe
failover, and knowing when a replica is "complete" - so if you bring up
a new replica, knowing when all the users have been replicated and
rolling replication is close enough that you can shut down and switch.

Bron.