Unfortunately we aren't capturing the UserAgent in any of our logs. That's something I'll want to adjust. On Tue, Sep 24, 2013 at 11:41 AM, john gale <john@xxxxxxxxxxxx> wrote: > > Sometimes client bugs can manifest itself this way, say if a malformed redirect continues redirecting them back to the same page, and a client bug prevents noticing this. > > What was the UserAgent of the user ? > > ~ john > > On Sep 24, 2013, at 10:37 AM, Ryan Merrell wrote: > >> We've had a problem occur maybe 5 times over the last year where a >> single user is DoS'ing our web server (unintentionally) and I'm having >> a hard time figuring out how it's happening. >> >> >> Background Info: >> >> We're an online school with moderate traffic levels (800+ unique IPs >> with 18,000 GET/POST over the last hour as of this post). >> We're running Apache/2.2.15 Release 15.el6_2.1 from RHEL. We're >> running RHEL 6.3. >> >> >> The few times that this has happened, we've looked at the offending >> user's traffic and noticed that before they DoS'd us, they were logged >> in doing their course work as expected. But then we would suddenly get >> thousands of GET requests for the home page, which showed up in the >> access logs like this: >> >> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 >> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 >> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496 >> >> In this last case, it was repeated over 7000 times in a span of about >> 18 minutes (generally around 10 requests per second). What ended up >> happening is that we reached our MaxServerLimit number and Apache >> eventually died. I've started looking at some tools, such as >> mod_evasive, to protect us from such problems. But then I got thinking >> about what was actually happening here, tried to reproduce the problem >> myself, but couldn't! I used JMeter to simulate a heavy attack (more >> connections at a faster rate -- around the ballpark of 25000 requests) >> and I never spawned more than 15 child processes or so. My test >> environment handled it perfectly. But something about how the student >> did it made httpd spawn children like crazy, which eventually killed >> it. >> >> Here are some of my server configs that I feel are relevant. Any >> advice on what is actually happening here and what I can do to >> alleviate the problem would be appreciated. >> >> KeepAlive On >> MaxKeepAliveRequests 100 >> KeepAliveTimeout 5 >> >> <IfModule prefork.c> >> StartServers 8 >> MinSpareServers 5 >> MaxSpareServers 20 >> ServerLimit 2000 >> MaxClients 2000 >> MaxRequestsPerChild 4000 >> </IfModule> >> >> >> >> >> Ryan Merrell >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx >> For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx > For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|