Re: Weird Ip in access log, need help to understand

André Warnier <aw@xxxxxxxxxx> · Thu, 19 Jun 2008 16:37:11 +0200

Artem Kuchin wrote:

André Warnier пишет:

Artem Kuchin wrote:

I recently upgraded to 2.2 and now i see thing which i have never 

seen mefore in

the access log:

74.6.29.159, 74.6.8.118 - - [11/Jun/2008:06:28:55 +0400] "GET 

/Go/ViewProduct/id=883 HTTP/1.1" 200 7195 "-" "Mozilla/5.0 

(compatible; Yahoo! Slurp; http://he

lp.yahoo.com/help/us/ysearch/slurp)"

Following the link to :
http://help.yahoo.com//help/us/ysearch/slurp
may help understand

no, it does not. The problem is not with slurp and is not related to any 

particular client. It is not even

a problem in a common meaning. The question is WHY in this record are 2 

(TWO) ip address in

the first field of the record: 74.6.29.159, 74.6.8.118

How it is possible at all? One request comes only from one ip. Where 

apache takes these

addresses?

Well ok.

Now both IP's are indeed  from yahoo's crawlers (of which there are 

probably many hundreds) :

dent:~# nslookup 74.6.29.159
Non-authoritative answer:
159.29.6.74.in-addr.arpa        name = lj512625.crawl.yahoo.net.
dent:~# nslookup 74.6.8.118
Non-authoritative answer:
118.8.6.74.in-addr.arpa name = llf520026.crawl.yahoo.net.

What I would guess is that Apache somehow regroups identical and 

simultaneous requests into a single log line.  And I would guess that 

Yahoo crawlers work "in teams", so that it is not surprising that 

several requests would arrive simultaneously to your server.

I'm just guessing here.

But it's interesting anyway.  I have not really looked specifically, but 

I don't recall seeing such lines on any of our servers.

A question : is the URL in those accesses, a real URL that exists on 

your site ?  I see a 200 OK return code, but some sites return a page 

even for an invalid URL.

If it is not, then there is one item on the Yahoo page that may explain 

why there are such quasi-simultaneous requests.  I think it's called 

"why is yahoo trying to get strange urls from my site ?"

André

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
  "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx