[users@httpd] authenticating robots by user-agent & double reverse lookup

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

To: <users@xxxxxxxxxxxxxxxx>
Subject: [users@httpd] authenticating robots by user-agent & double reverse lookup
From: "Ruth, Brice D" <BRUTH@xxxxxxxxx>
Date: Fri, 22 Sep 2006 09:03:48 -0500
Authentication-results: idunn.apache.osuosl.org smtp.mail=BRUTH@xxxxxxxxx; spf=permerror
Delivered-to: apmail-httpd-users-archive@xxxxxxxxxxxxxx
Delivered-to: apmail-httpd-users-archive@xxxxxxxxxxxxxxxx
Delivered-to: mailing list users@xxxxxxxxxxxxxxxx
Mailing-list: contact users-help@xxxxxxxxxxxxxxxx; run by ezmlm
Reply-to: users@xxxxxxxxxxxxxxxx
Thread-index: AcbeT+1RrIRcSOM8Tm+Dh/VB59Fo+g==
Thread-topic: authenticating robots by user-agent & double reverse lookup

Title: Message

Good morning.

After reading the latest Google Webmaster blog entry (http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html) I got to thinking about how one might be able to configure Apache deny visits from bots that purport to be Googlebot (or anything else) but do not pass the double reverse lookup for googlebot.com.

Reading the docs, it looks like mod_access will do a double reverse lookup on something like:

Allow from googlebot.com

And mod_access can also respect env variables set via

SetEnvIf User-Agent Googlebot

But, how would a webmaster go about combining these to stop scammers?

Thanks for any tips!

-Brice

Prev by Date: Re: [users@httpd] International File Names
Next by Date: Re: [users@httpd] International File Names
Previous by thread: [users@httpd] Apache with ExtFilterDefine problem
Next by thread: [users@httpd] AuthZ POST content and multiple handlers
Index(es):
- Date
- Thread

[Index of Archives] [Open SSH Users] [Linux ACPI] [Linux Kernel] [Linux Laptop] [Kernel Newbies] [Security] [Netfilter] [Bugtraq] [Squid] [Yosemite News] [MIPS Linux] [ARM Linux] [Linux Security] [Linux RAID] [Samba] [Video 4 Linux] [Device Mapper]