We've had a problem occur maybe 5 times over the last year where a single user is DoS'ing our web server (unintentionally) and I'm having a hard time figuring out how it's happening. Background Info: We're an online school with moderate traffic levels (800+ unique IPs with 18,000 GET/POST over the last hour as of this post). We're running Apache/2.2.15 Release 15.el6_2.1 from RHEL. We're running RHEL 6.3. The few times that this has happened, we've looked at the offending user's traffic and noticed that before they DoS'd us, they were logged in doing their course work as expected. But then we would suddenly get thousands of GET requests for the home page, which showed up in the access logs like this: [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 In this last case, it was repeated over 7000 times in a span of about 18 minutes (generally around 10 requests per second). What ended up happening is that we reached our MaxServerLimit number and Apache eventually died. I've started looking at some tools, such as mod_evasive, to protect us from such problems. But then I got thinking about what was actually happening here, tried to reproduce the problem myself, but couldn't! I used JMeter to simulate a heavy attack (more connections at a faster rate -- around the ballpark of 25000 requests) and I never spawned more than 15 child processes or so. My test environment handled it perfectly. But something about how the student did it made httpd spawn children like crazy, which eventually killed it. Here are some of my server configs that I feel are relevant. Any advice on what is actually happening here and what I can do to alleviate the problem would be appreciated. KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 <IfModule prefork.c> StartServers 8 MinSpareServers 5 MaxSpareServers 20 ServerLimit 2000 MaxClients 2000 MaxRequestsPerChild 4000 </IfModule> Ryan Merrell --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|