Re: robots.txt and rewrite rule

Norman Khine <norman@xxxxxxxxx> · Sat, 24 Jan 2009 16:20:03 +0100

Bob Ionescu wrote:

2009/1/23 Norman Khine <norman@xxxxxxxxx>:

    RewriteEngine On
    #DenyHosts Rules
    RewriteMap  hosts-deny txt:/home/user/txt/hosts.deny
    RewriteCond ${hosts-deny:%{REMOTE_HOST}|NOT-FOUND} !=NOT-FOUND [OR]
    RewriteCond ${hosts-deny:%{REMOTE_ADDR}|NOT-FOUND} !=NOT-FOUND [OR]
    RewriteCond ${hosts-deny:%{HTTP:true-client-ip}|NOT-FOUND} !=NOT-FOUND
    RewriteCond $1 !=robots.txt
    RewriteRule ^/.*  -  [F]

That's the wrong rule, it should be placed above the rule which proxies, i.e.

RewriteCond $1 !=robots.txt
RewriteRule ^/(.*) http://localhost:12080/companies/$1 [P]

Thanks, I corrected this, but now I get a 403 Forbidden page

If I access the http://IP/robots.txt it works fine.

$cat rewrite.log

86.219.32.244 - - [24/Jan/2009:18:46:57 +0100] 

[domain.com/sid#81ce720][rid#8240688/initial] (4) RewriteCond: 

input='NOT-FOUND' pattern='!=NOT-FOUND' => not-matched

86.219.32.244 - - [24/Jan/2009:18:46:57 +0100] 

[domain.com/sid#81ce720][rid#8240688/initial] (3) applying pattern 

'^/(.*)' to uri '/robots.txt'

86.219.32.244 - - [24/Jan/2009:18:46:57 +0100] 

[domain.com/sid#81ce720][rid#8240688/initial] (4) RewriteCond: 

input='robots.txt' pattern='!=robots.txt' => not-matched

86.219.32.244 - - [24/Jan/2009:18:46:57 +0100] 

[domain.com/sid#81ce720][rid#8240688/initial] (1) pass through /robots.txt

$cat error.log

[Thu Jan 22 18:23:51 2009] [error] (111)Connection refused: proxy: HTTP: 

attempt to connect to [::1]:12081 (*) failed

[Fri Jan 23 20:45:26 2009] [error] (111)Connection refused: proxy: HTTP: 

attempt to connect to [::1]:12080 (*) failed

[Sat Jan 24 18:42:53 2009] [error] [client 86.219.32.244] client denied 

by server configuration: /usr/htdocs

[Sat Jan 24 18:43:34 2009] [error] [client 86.219.32.244] client denied 

by server configuration: /usr/htdocs[Sat Jan 24 18:44:57 2009] [error] 

[client 86.219.32.244] client denied by server configuration: /usr/htdocs

[Sat Jan 24 18:46:57 2009] [error] [client 86.219.32.244] client denied 

by server configuration: /usr/htdocs

Cheers

Norman

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
  "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx