Peter Hoskin wrote:
Hi,
I have written a script to parse a large amount of XML data and insert
it into SQL... deals with approx 80,000 rows each time I run it. I
cannot successfully complete this script without running out of memory.
how much memory do you have set as the max?
do you have control over this value? (i.e. is it not locked down by an admin)
what are you doing with each row? (I imagine that effectively you
only need on row in memory at a time - in which case the memory requirements are
probably quite conservative)
show us your code if you can :-)
Is pcntl_fork suitable to overcome this?
it may offer one suitable solution. but I would suggest looking at your
code again before adding another level of complexity that may not bee needed
(I would suggest that forking would be a good idea to improve the performance
by allowing parallel processing but not to fix a crash perse)
I have been playing with pcntl_fork and became completely stuck trying
to select data from a database and spawn a new fork for each row. Later
I'll have to use something like shmop so each child knows what row its
working on, and will do a series of xml parsing based on supplied data.
I doubt you need shmop given that each child gets a complete copy of the
parent processes address space (i.e. all variables, etc that were defined in the
parent process will also be available in each child process)
there are some examples in the user comments on this page:
http://php.net/manual/en/function.pcntl-fork.php
a simple example of forking with a foreach this way would be appreciated
Regards,
Peter Hoskin
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
