Some timing info on populating mailboxes

[Ross Boylan]

I tested using a 540 message mailbox (12.6MiB total size), transferring
with UW-IMAP's mailutil command.  I report here the number of threads
and the message transfer rate in messages/second.  In column A each
thread writes to the same IMAP box; in column B each thread writes to
a different box
(though all are for the same user):
Threads  A        B
 1	18.8
 2	29.8	 32.3
 4	43.5	 51.2
 7	51.6	 75.8
20	---	160.5

Times are wall-clock times on a system that didn't have much else to
do.  Everything (the source and destination files, the mailutil and
cyrus processes) were on one machine with one physical disk, though
multiple partitions.

Clearly there is a speedup from parallelism, so my assumption that the
filesystem or the database were the limiting factors was at least
partly wrong.  The speedup is well under linear.

I'm still not thrilled by the performance, or my (in)ability to
pinpoint where the bottleneck(s) are.  CPU useage was minimal.

An earlier report on this list said 
sysctl -w net.inet.tcp.delayed_ack=0 
worked wonders.  There does not seem to be such a setting in my Linux
2.6.18 kernel.  The only thing that appeared promising was
sysctl -w net.ipv4.tcp_low_latency=1
which was enabled for all the tests.  The single thread test without
it had a rate of 18.6 messages/second (vs 18.8 with it).
(There were also tcp_fack and tcp_sack, but they were already on).
I'm no TCP expert.

I want to try different file systems, which I think may be more
responsive to some tuning parameters, but I have to figure out how to
do that.

More Details:
Western Digital WD2500JS SATA disk, rated at 7200 RPM and 300MB/s
transfer.
Intel D945PSN motherboard with 3Ghz P4.
Stock Debian GNU/Linux 2.6.18 kernel (686 flavor, which thinks it has
2 CPU's thanks to hyperthreading).
Partitions are Reiser 3 on top of LVM under EVMS.
cyrus-imapd-2.2 Debian package, version 2.2.13-9

Timings for the multiple thread/multiple boxes scenarios were mostly
averages of 3 trials; the variability was small.  Other numbers are
one-shot.

Time did not appear sensitive to the number of messages already in the
box; it might matter that the entire cyrus system has quite a lot of
mail in it (11GB in spool directory--virtually all for my account).

Ross Boylan