Re: Working with large datasets

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Jason Pruim
lists@xxxxxxxxxxxxxxxxxxxx



On Oct 10, 2011, at 5:27 PM, Thompson, Jimi wrote:

> I really think that you should try running it from the command line and see what the issues are.  Get both Apache and php out of the way.  I've seen some PHP scripts use up all the file handles (OS limit) even on a 64 bit server when they start doing complex things with data sets.  
> 
> If it works ok without PHP/Apache then you can start lookig at PHP and APache.  
> 
> ISOLATE the issue not complicate it....
> 
> My 2 cents,

Hi Jimi,

I've done it from the command line a few times, first time I run a simple: SELECT * FROM Main WHERE state="test"; it takes:  2 rows in set (1 min 44.20 sec)
 after that initial run it's pretty responsive, usually around 0.01 seconds. 

If I do a select based on the new-york state, which is all of my almost 9 million records after the initial run, it returns it all VERY quickly:  25000 rows in set (0.02 sec)

So commandline is running fine after the initial...  I'm leaning towards either a problem with web server, PHP setup, or my PHP code... I've used the code many times before but never on such a large dataset...

When I pull the pagination out completely.... It's pretty much the same result... fetching the info for new-york works just fine but not "test"

One thing I am noticing right now though is the fact that when I switch over to using the test state.... Right now it's not displaying anything... Not even able to view the source...

Okay... Enough rambling right now... Need to do some more checking before I can come to a conclusion :)


 
-- 
PHP Database Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php




[Index of Archives]     [PHP Home]     [PHP Users]     [Postgresql Discussion]     [Kernel Newbies]     [Postgresql]     [Yosemite News]

  Powered by Linux