> http://cyrus.brong.fastmail.fm/patches/cyrus-statuscache-2.3.8.diff > > This cuts back on meta IO traffic considerably for repeated SELECT > calls on mailboxes which are unchanged. We are in a similar boat, > and it makes a huge difference (also for some other clients) That should be "cuts back on meta IO traffic considerably for repeated STATUS calls on mailboxes which are unchanged", not SELECT calls. That's ok though, because the reality is that most clients do many, many more STATUS calls than SELECT calls. Basically every client that has a "updated unread count on mailboxes every 5 minutes" is calling STATUS on every mailbox every 5 minutes. If you're getting the "unseen" count, that means stepping through the cyrus.index file of every mailbox every 5 minutes, which if you've got lots of users, with large mailboxes, each with an IMAP client doing a status every 5 minutes on every mailbox, means you could be forcing re-reads of many gigabytes of data every few minutes. If your cache memory isn't big enough, your IO will jump enormously. Basically this patch significantly reduced the amount of read IO from our meta partitions. > > of passing FDs around between processes. I don't know how it works > > precisely, but it uses Socket::Pool which had a sourceforge project > > and doesn't appear to any more. My guess is that I'd rather not > > look at the code for fear of going totally insane. > > You can pass file descriptors over UNIX domain sockets, see SCM_RIGHTS. > Here is some code Google found -> > http://search.cpan.org/src/SAMPO/Socket-PassAccessRights-0.03/passfd.c It does use this underneath. It's not that magic (eg fork() automatically copies file descriptors from one process to another, why shouldn't there be an explicit way to do that between non-parent/child processes), though it's not that commonly used either, and the usage api is a bit weird (strange messages via unix sockets). I haven't seen many programs that use it since you have to be quite careful with it (eg you can't read any data from a socket before you read the file descriptor or it breaks), and there have been bugs with many different OS's in the past (even linux 2.4 had a very intermittent bug for a long time in our testing, which is why we had a "try this 3 times" loop on a bunch of code). In our case it is very useful, since our imappool daemon keeps track of the server + user + currently selected folder, we basically call the imappool daemon to get a socket for a server/user combination, and it returns us one (connects to the server if needed), and tells us the state (eg connected but not logged in, logged in, logged in and which folder is selected). It then "locks" that socket in it's pool so nothing else can use it (eg another web process). We then do whatever we need with the socket, and when done, we tell the imappool to "unlock" the socket, and we also tell it the new state it's in (eg which users/folder). That way, the imappool always has the socket (we don't need to pass it back), but ensures that only one process can use it at a time. The main issue is that the code has to be changed to explicitly contact the imappool to get a connection and free it when done, it's not transparent. > In our installation it is not possible to log on as cyrus or other admin > users from outside, I patched the imapd to check the caller's ip address - > and the firewall does the rest so that no forged packet (with an inside > sender ip coming from outside...) will trigger it. We use nginx as a frontend proxy. For every IMAP/POP login, it "calls out" to a process that allows you to authenticate and/or authorise the login (you can return any error message you want, and it gives you information like login name, password and ip address - http://wiki.codemongers.com/NginxMailCoreModule), and what backend server + port to connect to. It uses a small process pool + epoll to easily handle 10,000's of connections. http://blog.fastmail.fm/2007/01/04/webimappop-frontend-proxies-changed-to-nginx/ Rob ---- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
