On 12/7/06, Linus Torvalds <torvalds@xxxxxxxx> wrote:
Does anybody who is web-server clueful know if there is some simple front-end (squid?) that is easy to set up and can just act as a caching proxy in front of such an incompetent server?
Squid in "transparent reverse proxy" mode isn't a bad choice, although I don't know offhand whether it queues/clusters concurrent requests for the same URL in the way you want. I suggest the "transparent" deployment (netfilter/netlink integration) because you can slap it in with no changes to the origin server and yank it out again if you have a problem. The challenge is in getting conntrack to scale to a zillion concurrent sessions, but you could probably find someone in your crowd who knows something about that. :-) Ignore any documentation that talks about httpd_accel_*. Configuring transparent mode is a great deal simpler and saner in squid 2.6 than it used to be; you just add a "transparent" parameter to the http_port tag. With or without this tag, you set up what used to be called "accelerator mode" using some parameters to http_port and cache_peer, as described in http://www.squid-cache.org/mail-archive/squid-users/200607/0162.html. If transparent mode looks like the right thing for kernel.org, you might be interested in some netfilter hackery to offload part of the conntrack session lookup load to a front-end box that blocks DDoS and acts more or less as an L4 switch plus session context cache. I've been banging on a proof of concept implementation for a while, and am currently working on integrating against 2.6.19 by splitting nf_conntrack into front and back halves that interact via a sort of Layer 2+ header. I have no idea yet whether it will have any scalability benefit on dual-x86_64 class hardware (it was originally conceived for rigid cache architectures where the random access patterns of session lookups have drastic cache effects). Cheers, - Michael - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html