On 12/20/2012 07:36 AM, Paolo Cravero wrote:
Hello,
anyone is willing to comment on this Cyrus server load, or share theirs?
Virtual machine on vSphere 5 with 2x 2 GHz vCPU;
cyrus-imapd RPM 2.4.16;
kernel 2.6.18 x64_64;
4 GB RAM.
There are 150 simultaneous IMAP connections on average mostly coming
from a webmail (openXchange), with 240 imapd processes. No POP3. Just
one instance of cyrus. Messages come over LMTP at a rate of 1
msg/second, average of course.
Data and metadata partitions are local to the virtual machine (not NFS),
connected to a SAN via FC.
The system has 20 data partitions (and 20 metadata), for a total of
3.8TB, of which 550GB are used. Mailboxes (accounts) sum up to 2000.
What is bugging me is the system load that averages at 10 during peak
hours. CPU usage is equally shared between user/system/idle. As reported
by vmstat:
us sy id wa st
33 31 32 4 0
These figures obviously vary each second, but don't go too far during
peak hours.
/proc/meminfo reports:
MemTotal: 4043792 kB
MemFree: 34416 kB
Buffers: 305456 kB
Cached: 2505672 kB
SwapCached: 0 kB
Active: 2393176 kB
Inactive: 1083692 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 4043792 kB
LowFree: 34416 kB
SwapTotal: 4192956 kB
SwapFree: 4192724 kB
Dirty: 86608 kB
Writeback: 0 kB
AnonPages: 665816 kB
Mapped: 95900 kB
Slab: 445736 kB
PageTables: 55056 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 6214852 kB
Committed_AS: 1018640 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 265672 kB
VmallocChunk: 34359472247 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 2048 kB
The userbase is going to grow. Another group of 3500 users with 2TB/year
traffic is going to go on a dedicated cyrus server, similar to the one
detailed above.
Not having a direct experience on large and active userbase under Cyrus,
the load seems high compared to other solutions we know of. I would
appreciate any input about the apparent ability of the current server to
hold more users, or which debugging/characterization steps should be
undertaken to further optimize our asset.
The "Performance Notes" page has been studied and applied when possible.
Thank you for reading so far and Season's Greetings, if applies.
Paolo
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Dell R510
64G of RAM
2 x X5670 Intel (6 core) processors
H700 PERC RAID card. RAID1 config for each mount point. /var/imap on SAS
15K disks while mail spool on 2TB nearline SAS (SATA) drives.
Cyrus IMAP 2.4.17 from source
Linux kernel is 2.6.18-308.16.1.el5
We have 10000 active users with about 500 concurrent connections.
Our load average is around 3 normally but goes as high as 20 once in a
while. But even when at 20 the system is not noticeably slow.
I suggest more RAM so more of the filesystem will be cached. And a fast
disk subsystem helps as well.
begin:vcard
fn:Patrick Boutilier
n:Boutilier;Patrick
org:;Nova Scotia Department of Education
adr:;;2021 Brunswick Street;Halifax;NS;B3K 2Y5;Canada
email;internet:boutilpj@xxxxxxxxxxx
title:WAN Communications Specialist
tel;work:902-424-6800
tel;fax:902-424-0874
version:2.1
end:vcard
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
