Put a file named 'robots.txt' in the root of your Web directory.
Here's a link.
http://www.searchtools.com/robots/robots-txt.html
Joy Methew wrote:
hiii all....
we can download any site from "wget -r " options.
if i want to stop downloading of my site from web server how i can do
this???
--
veritatas simplex oratio est
-Seneca
Andrew Bacchi
Systems Programmer
Rensselaer Polytechnic Institute
phone: 518.276.6415 fax: 518.276.2809
http://www.rpi.edu/~bacchi/
--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list
