On Mon, Apr 28, 2008 at 11:31 PM, Eric Déchaux <Eric.Dechaux@xxxxxxx> wrote: > Dear all, > > > I have juste setup a Cyrus infrastructure based on VMware Virtual > Infrastructure 3.0.1 and I have a huge performance problem. > > The new infrastructure was sized for 120 000 mailboxes with 42 000 > maximum concurrent IMAPS sessions (neither POP3 nor IMAP available). The > important point is the concurrent sessions count rise from 6 000 to 42 > 000 in less than an hour, at the beginning of the day when everybody > starts to work. > > > The hardware is made of two Sun Microsystems X4600 chassis with 8 dual > core AMD Opteron 885, 64 Gb RAM, 2 dual port 4G QLogic HBA and 4 Intel > e1000 NIC (2 being used for ESX System Console and VMotion). LAN backend > is 100% Giga Ethernet, SAN backend is 100% 4 Gb FC. > > > The Virtual machines are running Debian Etch 4.0r3 AMD64 and Cyrus 2.2 > and are configured as follow : > > On each physical server : > > - 4 frontends : 2 vCPU, 8 Gb RAM, 8 Gb SWAP / Total 8 vCPU, 32 Gb RAM > - 3 backends : 4 vCPU, 8 Gb RAM, 8 Gb SWAP, ~500 Gb reizerfs storage > through a 550 Gb VMFS LUN / Total 12 vCPU, 24 Gb RAM, 1.5 Tb storage > - 1 webmail : 2 vCPU, 2 Gb RAM, 2 Gb SWAP > - 1 smtp : 2 vCPU, 2 Gb RAM, 2 Gb SWAP > - 1 mupdate : 2 vCPU, 2 Gb RAM, 2 Gb SWAP > > Total : 26 vCPU, 62 Gb RAM. > > > This infrastructure is the consolidation of an existing one that runs > Debian Sarge 3.0 i386 with 8 frontends and 5 backends. Each server is a > dual Intel Xeon 3.0 Ghz with 4 Gb RAM and 4Gb SWAP. > > > The older infrastructure can stand the 42 000 concurrent sessions, the > new one can't : I was expecting each frontend to be able to handle 5 500 > concurrent sessions but they are not. Around 3 000 / 3 500 concurrent > sessions the frontends begin to SWAP and are not more able keep up the load. At least you know your problem :-) SWAP should be easier to solve than IO or CPU problem. I suppose your old frontends where never (or rarely) swapping ! You have replaced 2 old frontents with 4Gb by 1 bigger with 8Gb ! Something in the new configuration make the new frontent swap! Do you use 64bits software, 64bits OS ? Are you sure all the 8Gb are available to your applications or are in use ? Can you give a list of all configuration and software changes between old and new configuration ? Regards. > > > I don't have any system log that complains about something and the only > Cyrus message I got is a DB4 warning about lockers. I think the DB4 in > question is the TLS sessions cache DB and in my case the number of > lockers can be as high as 8 000... > > > > Does somebody have any clue of what I could tune / tweak / configure > either on Linux or Cyrus side to make the solution work better ? > Is my initial sizing wrong and only solution is to add hardware ? > > > Many thanks. > -- > Eric > ---- > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > -- Alain Spineux aspineux gmail com May the sources be with you ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
