How's your server load?

[Patrick Boutilier]

On 12/20/2012 07:36 AM, Paolo Cravero wrote:
> Hello,
> anyone is willing to comment on this Cyrus server load, or share theirs?
>
> Virtual machine on vSphere 5 with 2x 2 GHz  vCPU;
> cyrus-imapd RPM 2.4.16;
> kernel 2.6.18 x64_64;
> 4 GB RAM.
>
> There are 150 simultaneous IMAP connections on average mostly coming
> from a webmail (openXchange), with 240 imapd processes. No POP3. Just
> one instance of cyrus. Messages come over LMTP at a rate of 1
> msg/second, average of course.
>
> Data and metadata partitions are local to the virtual machine (not NFS),
> connected to a SAN via FC.
> The system has 20 data partitions (and 20 metadata), for a total of
> 3.8TB, of which 550GB are used. Mailboxes (accounts) sum up to 2000.
>
> What is bugging me is the system load that averages at 10 during peak
> hours. CPU usage is equally shared between user/system/idle. As reported
> by vmstat:
>
> us sy id wa st
> 33 31 32  4  0
>
> These figures obviously vary each second, but don't go too far during
> peak hours.
>
> /proc/meminfo reports:
> MemTotal:      4043792 kB
> MemFree:         34416 kB
> Buffers:        305456 kB
> Cached:        2505672 kB
> SwapCached:          0 kB
> Active:        2393176 kB
> Inactive:      1083692 kB
> HighTotal:           0 kB
> HighFree:            0 kB
> LowTotal:      4043792 kB
> LowFree:         34416 kB
> SwapTotal:     4192956 kB
> SwapFree:      4192724 kB
> Dirty:           86608 kB
> Writeback:           0 kB
> AnonPages:      665816 kB
> Mapped:          95900 kB
> Slab:           445736 kB
> PageTables:      55056 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:   6214852 kB
> Committed_AS:  1018640 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:    265672 kB
> VmallocChunk: 34359472247 kB
> HugePages_Total:     0
> HugePages_Free:      0
> HugePages_Rsvd:      0
> Hugepagesize:     2048 kB
>
>
> The userbase is going to grow. Another group of 3500 users with 2TB/year
> traffic is going to go on a dedicated cyrus server, similar to the one
> detailed above.
>
> Not having a direct experience on large and active userbase under Cyrus,
> the load seems high compared to other solutions we know of. I would
> appreciate any input about the apparent ability of the current server to
> hold more users, or which debugging/characterization steps should be
> undertaken to further optimize our asset.
> The "Performance Notes" page has been studied and applied when possible.
>
> Thank you for reading so far and Season's Greetings, if applies.
> Paolo
>
>
> ----
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
> To Unsubscribe:
> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
>


Dell R510
64G of RAM
2 x X5670 Intel (6 core) processors
H700 PERC RAID card. RAID1 config for each mount point. /var/imap on SAS 
15K disks while mail spool on 2TB nearline SAS (SATA) drives.
Cyrus IMAP 2.4.17 from source
Linux kernel is 2.6.18-308.16.1.el5

We have 10000 active users with about 500 concurrent connections.

Our load average is around 3 normally but goes as high as 20 once in a 
while. But even when at 20 the system is not noticeably slow.

I suggest more RAM so more of the filesystem will be cached. And a fast 
disk subsystem helps as well.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: boutilpj.vcf
Type: text/x-vcard
Size: 286 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20121220/5dcdd12d/attachment.vcf