Storage Sizing: IOPS per mailbox

[Ian Batten]

On 08 Jan 09, at 1508, Blake Hudson wrote:

> -------- Original Message  --------
> Subject: Storage Sizing: IOPS per mailbox
> From: ram <ram at netcore.co.in>
> To: info-cyrus <info-cyrus at lists.andrew.cmu.edu>
> Date: Friday, January 02, 2009 10:40:17 PM
>> When sizing a storage device for a large cyrus server, the typical
>> question asked by storage vendors is what is the IOPS required per
>> mailbox
>> M$$ Exchange has this concept of IOPS. and they suggest 1.5 IOPS per
>> mailbox ( heavy users )
>>
>> If I use postfix and cyrus , on my imap server ( pure IMAP  
>> server .. All
>> spam filtering , outgoing mails , authentication etc happens on
>> different servers )
>>
>>
>> If the storage is used only for imap storage , what is the typical
>> "IOPS" requirement per user
>> We will probably assume 30-50 mails a day of average 100k , and an  
>> email
>> client checking for new mail every 5minutes
>>
>>
>>
> In my experience I would estimate 0.1 IOPS per user for heavy users
> (thousands of emails per day, checked every few minutes) and 0.01 IOPS
> per user for typical ISP accounts (under a dozen emails, checked a few
> times daily).

Our IMAP server has as I type 1020 imap connections up, representing  
most of our staff.   The metadata (both /var/imap and per-mailbox) is  
in a ZFS pool configured as a two-way mirror of two-way stripes of SAS  
drives.  The load on that is low.  The zfs statistics on the metadata  
are as an example:

mailhost-new# zpool iostat 1
                capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
pool1       54.4G  23.6G     12     77   489K   499K
pool1       54.4G  23.6G      0    785      0  3.74M
pool1       54.4G  23.6G      0     14      0   135K
pool1       54.4G  23.6G      0     24      0   877K
pool1       54.4G  23.6G      0     33      0   242K
pool1       54.4G  23.6G      0     10      0  43.6K
pool1       54.4G  23.6G      0    417  1.48K  2.06M
pool1       54.4G  23.6G      0     22      0   139K
pool1       54.4G  23.6G      0      1      0  7.92K
pool1       54.4G  23.6G      0      7      0  31.7K
pool1       54.4G  23.6G      0     20      0  83.2K
pool1       54.4G  23.6G      0    504   1013  1.93M
pool1       54.4G  23.6G      0     23      0   574K
pool1       54.4G  23.6G      2     17  96.5K   123K
pool1       54.4G  23.6G      0     40      0   285K
pool1       54.4G  23.6G      0     26      0   123K
pool1       54.4G  23.6G      0    698  1.98K  3.41M
pool1       54.4G  23.6G      0      3      0  15.8K
pool1       54.4G  23.6G      0     24      0   744K
pool1       54.4G  23.6G      0     16      0   713K
pool1       54.4G  23.6G      3     15   209K   147K
pool1       54.4G  23.6G      5    569   760K  2.71M
pool1       54.4G  23.6G      0     16      0   222K
^Cmailhost-new#

You can see the five-second sync.  The first line, average, figures  
aren't representative because they of course include backup activity.

The actual messages are stored in the lowest (`archive') QoS band of a  
Pillar Axiom 500, in NAS mode.  The load is very small on each of two  
2TB-ish partitions (it's approaching four pm, so the business is going  
at close to full load):

mailpool1

I/O Operations
Read I/Os per second: 58.677
Write I/Os per second: 7.129
Average Request Time: 4.475 ms
Current MB per second: 1.003
General Statistics
Read/Write Cache Hit Percentage: 69%
Read/Write I/O Ratio: 89:11

mailpool2

I/O Operations
Read I/Os per second: 46.733
Write I/Os per second: 9.467
Average Request Time: 1.923 ms
Current MB per second: 0.544
General Statistics
Read/Write Cache Hit Percentage: 56%
Read/Write I/O Ratio: 83:17