Re: Single User DoS. How is this happening?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Unfortunately we aren't capturing the UserAgent in any of our logs.
That's something I'll want to adjust.

On Tue, Sep 24, 2013 at 11:41 AM, john gale <john@xxxxxxxxxxxx> wrote:
>
> Sometimes client bugs can manifest itself this way, say if a malformed redirect continues redirecting them back to the same page, and a client bug prevents noticing this.
>
> What was the UserAgent of the user ?
>
>         ~ john
>
> On Sep 24, 2013, at 10:37 AM, Ryan Merrell wrote:
>
>> We've had a problem occur maybe 5 times over the last year where a
>> single user is DoS'ing our web server (unintentionally) and I'm having
>> a hard time figuring out how it's happening.
>>
>>
>> Background Info:
>>
>> We're an online school with moderate traffic levels (800+ unique IPs
>> with 18,000 GET/POST over the last hour as of this post).
>> We're running Apache/2.2.15 Release 15.el6_2.1 from RHEL. We're
>> running RHEL 6.3.
>>
>>
>> The few times that this has happened, we've looked at the offending
>> user's traffic and noticed that before they DoS'd us, they were logged
>> in doing their course work as expected. But then we would suddenly get
>> thousands of GET requests for the home page, which showed up in the
>> access logs like this:
>>
>> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496
>> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496
>> [18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496
>>
>> In this last case, it was repeated over 7000 times in a span of about
>> 18 minutes (generally around 10 requests per second). What ended up
>> happening is that we reached our MaxServerLimit number and Apache
>> eventually died. I've started looking at some tools, such as
>> mod_evasive, to protect us from such problems. But then I got thinking
>> about what was actually happening here, tried to reproduce the problem
>> myself, but couldn't! I used JMeter to simulate a heavy attack (more
>> connections at a faster rate -- around the ballpark of 25000 requests)
>> and I never spawned more than 15 child processes or so. My test
>> environment handled it perfectly. But something about how the student
>> did it made httpd spawn children like crazy, which eventually killed
>> it.
>>
>> Here are some of my server configs that I feel are relevant. Any
>> advice on what is actually happening here and what I can do to
>> alleviate the problem would be appreciated.
>>
>> KeepAlive On
>> MaxKeepAliveRequests 100
>> KeepAliveTimeout 5
>>
>> <IfModule prefork.c>
>> StartServers       8
>> MinSpareServers    5
>> MaxSpareServers   20
>> ServerLimit      2000
>> MaxClients       2000
>> MaxRequestsPerChild  4000
>> </IfModule>
>>
>>
>>
>>
>> Ryan Merrell
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
>> For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
> For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx





[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux