Re: tuning question (Apache Users)

On Sat, Jul 12, 2014 at 1:25 PM, Miles Fidelman <mfidelman@xxxxxxxxxxxxxxxx> wrote:

Hi Folks,

Ever once in a while, a crawler comes along and starts indexing our site - and in the process pushes our server's load average through the roof.

Short of blocking the crawlers, can anybody suggest some quick tuning adjustments to make, to reduce load (setting the max. number of servers and/or requests, renicing processes)?

Use robots.txt to block access to dynamically generated resources which are
expensive to generate and not necessary for search hits?

Is it using a lot of concurrent requests, or is the main load issue due to

the cost of the requests it is making?

Yes - my next step is to go pour through manuals - but I expect others have done this enough to be able to point me at a few specific config file lines to change, and specific commands for identifying and renicing processes.

Thanks very much,

Miles Fidelman

--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx

Born in Roswell... married an alien...
http://emptyhammock.com/