LARGE single-system Cyrus installs?

[Vincent Fox]

This thought has occurred to me:

ZFS prefers reads over writes in it's scheduling.

I think you can see where I'm going with this.  My WAG is something
related to Pascal's, namely latency.  What if my write requests to 
mailboxes.db
or deliver.db start getting stacked up, due to the favoritism shown to 
reads?
The actual usage mix on a Cyrus system ends up being more writes than reads.
Despite having channels that seem under-utilized perhaps the stacking of
small latencies hits a tipping point that is causing the slowdown.

We have all Cyrus lumped in one ZFS pool, with separate filesystems for
imap, mail, sieve, etc.  However, I do have an unused disk in each array
such that I could setup a simple ZFS mirror pair for /var/cyrus/imap so
that the databases are in their own pools.  Or even I suppose a UFS
filesystem with directio and all that jazz set.  Perhaps I wouldn't get 
all the
effects of a RAM-SAN 500 but it could be a worthwhile improvement.

I liked having one big pool, but if it works, c'est la vie.