RE: CURL cannot connect to URL - IP address - after successful connection

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



> -----Original Message-----
> From: ioannes@xxxxxxxxxxxxxx [mailto:ioannes@xxxxxxxxxxxxxx]
> Sent: Monday, April 26, 2010 7:10 AM
> To: Tommy Pham
> Subject: Re:  CURL cannot connect to URL - IP address - after
successful
> connection
> 
> 
> 
> On 2010/04/27 1:13, Tommy Pham wrote:
> >> -----Original Message-----
> 
> > I assume that you did full testing with the browser as I suggested?
> > If everything works, one other thing to keep in mind is that the
> > target also may implement reverse DNS lookup in their anti-bot.  One
> > good way to test that is to remote in via SSH (if on Linux/Unix) to test
with
> wget.
> > Otherwise, I'm pretty sure that target site have some anti-bot
> > mechanisms in place.  Microseconds of analyzing valid 'user' requests
> > is better than processing 2-3 seconds and sending the response which
> > will consume bandwidth.  What you could also try is setting different
> > user-agents for every request and use cookies in cURL should the
> > target site have an anti-bot mechanism.
> >
> > Regards,
> > Tommy
> >
> >
> Yes, I think I have tested with/without cookies on the browser, trying
> different user agents (code emailed previously using array and rand) and
> cookies are used in script/not used.

And it works on subsequent requests?

> 
> I cannot work out how to use Putty/ssh/public private keys etc..wget...
> 

Learning how to use that is easier than learning to code PHP, IMO.
 
> I read about some problems with curl setting the port and a required patch
> on the server.
> 
> John

If cURL requires some kind of patch as you say, then it wouldn't have work
in the first place.  Perhaps it's better to post your (obfuscated personal
data) code.  Or try on your local PC on your local web server to eliminate
possibility of proxies, anti-bots, etc... problems to test that your code
works as intended and not cURL problem as you say.  I didn't have problems
using cURL before.  But then my targeted sites were very big companies and
didn't care about bots much.

Regards,
Tommy

PS:  Always reply to the list so others in the future can benefit unless
it's something personal.


-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux