Re: page suck attack

[Michael McGlothlin <michaelm@xxxxxxxxxxx>]

robert wrote:
> Hi
> Every so often my site is "attacked" in which all URLS on my site are 
> retrieved in the span of minutes. What is this called?? I mean what do 
> I google for? I don't know where to begin.
>
> I'm not sure if I am going to implement such at thing but I would like 
> to be able to research it to know my options.
>
> thank you
> robert
Each of our websites get a page sucked about every 5 seconds from Google 
alone. Sometimes seems a little aggressive but I'd rather they be 
indexing our websites than not. Other spiders are usually more moderate 
in my experience. I usually make sure my robots.txt blocks downloading 
css, scripting, images, and files by spiders so at least all they're 
getting is the HTML content that is actually useful.

--
Michael McGlothlin
Southwest Plumbing Supply

<<attachment: smime.p7s>>