Re: CURL cannot connect to URL - IP address - after successful connection

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





Just to eliminate all possibilities, are you to open the same URL/URI in
the
web pages repeatedly?  Also, what happens when you fake the user agent in
the web browser?  The target site may have some anti bot mechanism in
place to reduce stress/load on the server(s).

Regards,
Tommy

One more thing, check it with cookies enabled/disabled in the web browser
too.



Having deleted cookies on the browser and disabled them, it still does not like various user agents:

$useragent = array('Mozilla','Opera','Microsoft Internet Explorer','ia_archiver'); $os = array('Windows','Windows XP','Linux','Windows NT','Windows 2000','OSX');
	//random user agent code
$agent = $useragent[rand(0,3)].'/'.rand(1,8).'.'.rand(0,9).' ('.$os[rand(0,5)].' '.rand(1,7).'.'.rand(0,9).'; en-US;)';
	//would give something like Mozilla/3.5 (Windows 5.4; en-US;)
	
-- OR --

//$useragent='Google Image - Googlebot-Image/1.0 ( http://www.googlebot.com/bot.html)'; //$useragent="MSN Live - msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)";

--  OR --
//$agent = "DocZilla/1.0 (Windows; U; WinNT4.0; en-US; rv:1.0.0) Gecko/20020804";

I am just calling the page manually, once at a time. It is probable that there is some anti-bot measures. Page would probably not want to be indexed as it is providing ever changing content. How to use this for normal level of use for real user just in a different site?

John

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux