LARGE single-system Cyrus installs?

[Bron Gondwana]

On Thu, Nov 15, 2007 at 01:29:54PM -0500, Wesley Craig wrote:
> On 14 Nov 2007, at 23:15, Vincent Fox wrote:
> > We have all Cyrus lumped in one ZFS pool, with separate filesystems  
> > for
> > imap, mail, sieve, etc.  However, I do have an unused disk in each  
> > array
> > such that I could setup a simple ZFS mirror pair for /var/cyrus/ 
> > imap so
> > that the databases are in their own pools.  Or even I suppose a UFS
> > filesystem with directio and all that jazz set.
> 
> About 30% of all I/O is to mailboxes.db, most of which is read.  I  
> haven't personally deployed a split-meta configuration, but I  
> understand the meta files are similarly heavy I/O concentrators.

Which is a good argument for checkpointing it (gah, hate that term -
it's so non-specific.  I've spent some time working on terminology
maps for this stuff, and "repack" is the current winner, mainly due
to be shorter than the runner up "consolidate")

What was I saying again?  Oh - yes.  Current skiplist metric is that
the mailboxes.db has to be be twice the size of the last checkpointed
size plus 16k before it re-checkpoints.  Given that a checkpoint takes
approximately 2 seconds on our systems, and it means that you're not
seeking all over the place any more, it would almost certainly be a
win.

That said, we don't have a single machine where the memory pressure
is tight enough to ever push mailboxes.db out of the cache, so it's
not ever going to be hitting the disk for reads anyway!

Bron.