On Thu, June 23, 2005 3:37 am, JamesBenson said: > http://www.funender.com/phpBB2/about18577.html Call me crazy, but... A) Doesn't robots.txt have to be in public_html? How the hell can the robots read it if it's in the root folder, as they suggest in that forum? B) Are the Bad Bots really going to honor robots.txt in the first place? I mean, I know the spammers don't even need to un-obfuscate emails as simple as %40 and #&64; but are they really so dumb as to honor robots.txt while they write their crawlers?... That's crazy. Though I guess my theory on that holds true here as well: If you catch a million fish every time you cast your line, are you gonna try better bait? No. So they don't un-obfuscate even the simplest email mask, and I guess some of them play nice as far as robots.txt goes, even though they are out to spam you. Seems silly, but that could be Reality. C) Aren't some of those "bad" bots also going to rule out legitimate scripting? Am I mistaken that he pre-emptorally denies access to any (legit) Python script? Hello? I'm not a fan of Python, but that seems a bit much. :-) [Just kidding. I'm ambivalent towards Python, really.] -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
