Re: Re:Re: [users@httpd] how to block the duplicated requests?

[Kurtis Rader <krader@xxxxxxxxxxxxx>]

On Tue, May 19, 2015 at 8:22 AM, javalishixml <javalishixml@xxxxxxx> wrote:

I see you speak "Though those modules exist, this might be best implemented in your application"

How can I understand it? I should write a c program for apache? or I should write some java web application at tomcat level?

Because this is a web site, it has to face concurrency issue. So I prefer to resolve this issue at httpd level.
Not sure if my thinking is good or not? 

Yehuda is correct. You should implement the rate limiting in your lottery code, not the web server. You are rate limiting lottery attempts, not HTTP requests. Nor are you eliminating duplicate requests. If you've never written a rate limiter the concept is fairly straightforward. Google "token bucket". Below is a simple implementation I use in my httpd and sshd log monitoring app to detect malware that is flooding my server with bogus requests. It uses a single token bucket and is thus much simpler and uses less memory than more flexible implementations.

Here is an example of how it is used:

blackhole_source = False

if web_req.http_method == 'POST':

    rate_limiter = self.rate_limiter_post.get(ip_addr, None)

    if rate_limiter is not None:

        blackhole_source = rate_limiter()

    else:

        # Allow no more than two POST requests every five seconds.

        # In my experience legitimate WordPress administration doesn't

        # exceed this limit.

        self.rate_limiter_post[ip_addr] = util.RateLimiter(2, 5)

if blackhole_source:

And here is the implementation:

class RateLimiter():

    """Return true if the allowable rate has been exceeded."""

    # pylint: disable=too-few-public-methods

    def __init__(self, rate, per):

        """Initialize object for a specific situation (e.g., IP address).

        Note that rate and per must have the same temporal units; e.g., seconds.

        """

        if not isinstance(rate, int) or not isinstance(per, int):

            raise TypeError('rate and per must be integers')

        self.rate = float(rate)

        self.per = float(per)

        self.ratio = self.rate / self.per

        self.allowance = float(rate)

        self.last_check = time.time()

    def __call__(self):

        """Return True if the rate has been exceeded."""

        now = time.time()

        time_delta = now - self.last_check

        self.last_check = now

        self.allowance += time_delta * self.ratio

        if self.allowance > self.rate:

            self.allowance = int(self.rate)  # throttle

        if self.allowance < 1.0:

            return True

        else:

            self.allowance -= 1.0

            return False

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank