On Fri, Aug 04, 2006 at 03:26:39PM +1000, Robert Mueller wrote: > Anyway the good news: > Before: 2 frontend servers with 7000+ connections (eg 14,000+ total) using > 6G of RAM with a load on each of about 2 > After: 1 frontend server with 14,000+ connections, less than 1G of RAM > usage, load of under 0.5 > > The only problem is that all the docs are in Russian. I should get around > to writing up a document on setting it up, it's not that hard. The main > issue is that you have to write your own authentication daemon. It doesn't > connect to anything standard like mysql or LDAP, but it does use a simple > protocol you can write your own adaptor for quickly in something like perl. Actually, one other scalability issue you should be aware of (and we'll have to actually look into again soon...) Each backend imap server is only listening on one address:port, e.g. in our case my imap server listens on 10.x.x.5:2143 internally. Your frontend only can make connections out using any port it likes, but there are only 65k of them, and at any one time, a fraction of those will be tied up doing other things. We actually had to set a TCP parameter on our backends (net.ipv4.tcp_rfc1337 = 0) so that when a busy frontend re-used a socket pair for a previously closed connection it didn't just get dropped immediately and return a "connection failed" back to nginx and hence the client. That's fine for now, but if we ever hit 65k connections from a frontend to a single imap server (eek) then we suspect that it will either lock up trying to bind the connection or nginx will fail back to the client, not a good look. (my just-this-second thought involves multiple listening ports on the IMAP backends with multiple imapd lines in cyrus.conf and a randomiser in the nginx authentication proxy, but I haven't talked to Rob about that yet... since I only came up with it as I wrote this!) Bron. ---- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
