RES: CURL problems still

[Jônatas Zechim <zechim.php@xxxxxxxxx>]

Hey Guy, try to change the 'max_execution_time'

ini_set('max_execution_time','1800');



-----Mensagem original-----
De: Miller, Terion [mailto:tmiller@xxxxxxxxxxxxxxxxxxxx] 
Enviada em: segunda-feira, 11 de maio de 2009 16:41
Para: PHP-General List
Assunto:  CURL problems still 

I'm back.  I thought I had this script working but it seems to always stop
at the end of the C's, so I can get it to go thru pages A-C then it stops, I
have tried setting the time-out to a very large amount but it never
continues to run past the same spot, it gathers all records to the end of C,
then quits...any ideas?

Script: 

<?php

ini_set('display_errors',
1);
include("inc/dbconn_open.php");

include("inc/dom.php");

error_reportin
g(E_ALL);

$TESTING = TRUE;


 $targets["a"] =
"http://www.greenecountymo.org/sheriff/warrants.php?search=A";;

$targets["b"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=B";;

$targets["c"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=C";;

$targets["d"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=D";;

$targets["e"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=E";;

$targets["f"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=F";;

$targets["g"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=G";;

$targets["h"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=H";;

$targets["i"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=I";;

$targets["j"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=J";;

$targets["k"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=K";;

$targets["l"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=L";;

$targets["m"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=M";;

$targets["n"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=N";;

$targets["o"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=O";;

$targets["p"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=P";;

$targets["q"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=Q";;

$targets["r"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=R";;

$targets["s"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=S";;

$targets["t"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=T";;

$targets["u"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=U";;

$targets["v"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=V";;

$targets["w"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=W";;

$targets["x"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=X";;

$targets["y"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=Y";;

$targets["z"] = 
"http://www.greenecountymo.org/sheriff/warrants.php?search=Z";;
 
 foreach
($targets as $target_url){

echo $target_url;

$userAgent = 'Googlebot/2.1
(http://www.googlebot.com/bot.html)';


$ch = curl_init();
curl_setopt($ch,
CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch,
CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR,
true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch,
CURLOPT_AUTOREFERER, true);
curl_setopt($ch,
CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT,
6000);

$html = curl_exec($ch);

if (!$html) {
    echo "<br />cURL error
number:" .curl_errno($ch);
    echo "<br />cURL error:" . curl_error($ch);

exit;
}


// Create DOM from URL or file
$html =
file_get_html($target_url);

// Find table
foreach($html->find('table') as
$table)
{
    foreach($table->find('tr') as $tr)
    {
       
       

// Grab children
       $cells = $tr->children();
              

if($cells[0]->plaintext != "Name")
        {
        
            for ($i =
0; $i < count($cells); $i++)
            {
                 

switch ($i){
                 case 0: // Name
                     $name =
$cells[$i]->plaintext;
                     echo $cells[$i]->plaintext;

break;
                 case 1: // Age
                     $age =
$cells[$i]->plaintext;
                     break;
                 case 2:
// Warrant type
                     $warrant = $cells[$i]->plaintext;

break;
                 case 3: // Bond amount
                     $bond =
$cells[$i]->plaintext;
                     break;
                 case 4:
// Warrant number
                     $wnumber = $cells[$i]->plaintext;

break;
                 case 5: // Offence description

$crime = $cells[$i]->plaintext;
                     break;

Default:
                     echo "Uh-oh<br />";
                 }

                 
                 
            }
            
           

        }
       
    
         // Build your INSERT statement here

    // Build your INSERT statement here
      
        $query = "INSERT
into warrants (wid, name, age, warrant, bond, wnumber, crime) VALUES (";

$query .= " '$wid', '$name', '$age', '$warrant', '$bond', '$wnumber',
'$crime' )";
        //$wid = mysql_insert_id();
        
        echo
$query;
        
        // run query
     mysql_query($query) or die
(mysql_error());
    }
       
      }
        
    
}


?>



-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php