Re: stop web spiders

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Anze Vidmar wrote:
Is it possible to stop spiders to access specific folder on my web
server? e.g. http://myweb.com is public, but
http://myweb.com/confidential has some confidential files in it and I
don't want them to show while searching google for example. The password
protection (.htaccess) is *not* an option. I have "-Indexes" option set
on this folder, but still not sure if spiders can read them or not?


Look into robots.txt

Regards,

Sean

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list

[Index of Archives]     [CentOS]     [Kernel Development]     [PAM]     [Fedora Users]     [Red Hat Development]     [Big List of Linux Books]     [Linux Admin]     [Gimp]     [Asterisk PBX]     [Yosemite News]     [Red Hat Crash Utility]


  Powered by Linux