* Brian Dunning <brian@xxxxxxxxxxxxxxxx> : > On Aug 11, 2005, at 4:06 PM, Evert | Collab wrote: > > > First hit on google: > > http://www.searchengineworld.com/robots/robots_tutorial.htm > > Search engines check for a robots.txt on your site, in the > > robots.txt file you can specify that certain or all search engines > > shouldn't index your site > > I know what robots.txt is, I meant how would you use that to cloak > the site. Put PHP code in robots.txt to log the IP of any requests > to a db, and then use that db to cloak the rest of the site or not? If you want to dynamically determine what to disallow based on the UserAgent string, simply tell Apache, via an .htaccess file, to pass robots.txt to PHP for handling. Then have that script do the processing and return output compatible with the robots.txt specification. -- Matthew Weier O'Phinney Zend Certified Engineer http://weierophinney.net/matthew/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
