On Fri, September 15, 2006 10:42 am, Matthew H. North wrote: > We're developing a web application that involves traversal of a > hierarchical database structure (MySQL, PEAR::DB, and > PEAR::DB::DataObject). Currently that traversal is done recursively, > and involves visiting thousands of nodes in the tree. However, the > tree is relatively flat, and the recursion never gets more than 4 or 5 > calls deep. A severely truncated but illustrative version of the code > of interest is: So you are just visiting the nodes, and not doing anything with them? It's entirely possible that PEAR::DB and/or DataObject are trying to cache something to "help" you... You should be able to quickly hack a *BAD* page of code with minimal error checking to do whatever queries PEAR::DB is doing for you. Take those out of the picture with just mysql_query() calls to see if it's in your code, or in PEAR code. This will at least break the problem into the "my code" versus "their code" category. For some web apps, the advantages of PEAR::DB and friends is pretty thin... > trigger_error(memory_get_usage()); > $result = traverse_hierarchy(); At crucial points within the hierarchy, perhaps at nodes/leaves you expect to be at specific milestones (halfway, 25%, 75%, ...) start adding code that does crude things like: if ($node->name == 'This one node we think is halfway through') trigger_error(memory_get_usage()); Log the numbers into a db with the node names and then later graph it to see if the memory is getting chewed up in a straight line or if it jumps at some point. If there's a big jump somewhere, you know where to look. If it's a straight line, then you can start doing the same thing line by line to find where the RAM is going. > 2500000 > 8000000 > (key):(memory released) > . > . > . > 5500000 > > After unsetting ALL globals within the global context (there are no > stack frames present, and therefore no non-global variables), PHP > still claimed my script was holding onto 5.5MB!! Did you close down the DB connection and kill the PEAR objects?... PHP's garbage collection has had... issues... in the past. > The question is this: Given the following assumptions: > > 1) PHP's memory manager reclaims memory when all references to that > memory are > gone. Well, it tries to anyway... It's not always that simple, particular with variable variables and other dynamic features. > 2) A reference is 'gone' when it goes out of scope or is 'unset'. Scope seems like it should be simple, but it's not. Use unset to be certain. > 3) The only references that remain in the global context are > references to globals (all non-global variables have gone out of scope > and that memory reclaimed) See #1. PHP "scope" is not as clean-cut as C. A simple "for" loop in PHP leaves the iterator variable, last I checked. Inside a function, that should go out of scope. Outside a function, it stays around. foreach, I think, correctly un-scopes the vars. > 4) $GLOBALS is a PHP special associative array that contains the name > and > value of all global variables. Yes. > 5) By doing unset($GLOBALS[$varname]) and unset($$varname), where > $varname > is > each key of the $GLOBALS array, I am effectively eliminating all > remaining > references, and all allocated memory should be reclaimed by the memory > manager (except perhaps for memory associated with function and class > definitions). No. Dangling pointers and references not correctly cleaned up from a function are left out in limbo. > 6) Resources (think database resources) are automatically freed by > garbage collection when there are no more references to them Probably, eventually, if PHP's GC kicks in when you think it does. To be certain, close the DB references when you are done with them. > 7) No additional code is being evaluated within traverse_hierarchy > 8) I'm correct that there aren't any circular references in my code > nor in any PEAR module code Circular references are not a problem, really. It's the ones that get chopped off from any connection to anything you can get ahold of and start releasing that matter. And if you get a whole big chain of them, with no root to tie onto to start releasing... > Are there any other ways that user code can result in this apparent > memory leak situation? If so, what are they? PHP Extensions can have a memory leak. Ain't much PHP can do about that, really. > Or, are any of my first 6 assumptions incorrect? They're a little too optimistic. :-) In the meantime, of course, just crank up your memory_limit to 16M -- It's going to be the default in PHP6 anyway, and many many many web apps these days need 16M just to start. Not that you want to ignore the problem, but you've got better things to do than chase this down in fire drill mode if your servers can handle 16M limit instead of 8M. If you're talking super high-volume high-load site, maybe this IS priority #1. But for MOST web apps, on MOST common usage, cranking to 16M instead of 8M to "solve" the problem, as irritating as it might be to your purist soul, is the Right Answer. You'll never notice the difference, it won't cost you a dime, and you'll be able to move on to the next task. Long-term, you want to find out what you are doing to chew up 5.5M for nothing, and stop doing that, of course, as it WILL matter some day. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
