On 5/2/05, Abu Hurayrah <abu_hurayrah@xxxxxxxxxxxxx> wrote: > Greets to all! > > I've just noticed that when the MSNbot crawls my website and hits some of my > downloads, it doesn't download the whole file. Most search engines are only interested in the first x bytes of the file, so the bot may simply be dropping the connection after it gets what it want.s > MSN seems to only catch ONE chunk, no matter what size I make it, which I > find very strange, because I cannot think of why my implementation would > matter to MSN or not. What is the smallest "chunk" size you have tried? You are probably just not detecting the dropped connection until you have sent a chunk, so you don't really knwo what the bot is accepting. Joshua. --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|