Philip Hallstrom wrote:
robots.txt will not do what you want it to.
Just sniff for those robots' User-Agents (Google, MSN and Yahoo all
publish their UA strings on their websites, AFAIK) and send different
content if it's one of those.
they will hammer you for it eventually - AFAICT all major SEs send out
their
spiders occasionally with faked user-agent strings - to catch out crap
like
this.
google adsense won't. I explicity asked them about this. Well, what I
asked was that if I had a password protected area, could I allow them
access to spider the content so that normal users could see the ads. I
told them the layout would be different, but the content the same.
They said that was fine.
but you didn't ask - 'heh is it okay to fill my public page with SEO crud but
only if a spider comes round'
they might just take a different view on that :-)
2cents.
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
