> -----Original Message----- > From: Jim Giner [mailto:jim.giner@xxxxxxxxxxxxxxxxxx] > Sent: Monday, September 02, 2013 8:14 PM > To: php-general@xxxxxxxxxxxxx > Subject: Re: refernces, arrays, and why does it take up so much > memory? > > On 9/2/2013 9:30 PM, Daevid Vincent wrote: > > I'm confused on how a reference works I think. > > > > I have a DB result set in an array I'm looping over. All I simply want to > do > > is make the array key the "id" of the result set row. > > > > This is the basic gist of it: > > > > private function _normalize_result_set() > > { > > foreach($this->tmp_results as $k => $v) > > { > > $id = $v['id']; > > $new_tmp_results[$id] =& $v; //2013-08-29 [dv] using > a > > reference here cuts the memory usage in half! > > unset($this->tmp_results[$k]); > > > > /* > > if ($i++ % 1000 == 0) > > { > > gc_enable(); // Enable Garbage Collector > > var_dump(gc_enabled()); // true > > var_dump(gc_collect_cycles()); // # of > elements > > cleaned up > > gc_disable(); // Disable Garbage Collector > > } > > */ > > } > > $this->tmp_results = $new_tmp_results; > > //var_dump($this->tmp_results); exit; > > unset($new_tmp_results); > > } > > > > Without using the =& reference, my data works great: > > $new_tmp_results[$id] = $v; > > > > array (size=79552) > > 6904 => > > array (size=4) > > 'id' => string '6904' (length=4) > > 'studio_id' => string '5' (length=1) > > 'genres' => string '34|' (length=3) > > 6905 => > > array (size=4) > > 'id' => string '6905' (length=4) > > 'studio_id' => string '5' (length=1) > > 'genres' => string '6|37|' (length=5) > > > > However it takes a stupid amount of memory for some unknown reason. > > MEMORY USED @START: 262,144 - @END: 42,729,472 = 42,467,328 BYTES > > MEMORY PEAK USAGE: 216,530,944 BYTES > > > > When using the reference the memory drastically goes down to what I'd > EXPECT > > it to be (and actually the problem I'm trying to solve). > > MEMORY USED @START: 262,144 - @END: 6,029,312 = 5,767,168 BYTES > > MEMORY PEAK USAGE: 82,051,072 BYTES > > > > However my array is all kinds of wrong: > > > > array (size=79552) > > 6904 => & > > array (size=4) > > 'id' => string '86260' (length=5) > > 'studio_id' => string '210' (length=3) > > 'genres' => string '8|9|10|29|58|' (length=13) > > 6905 => & > > array (size=4) > > 'id' => string '86260' (length=5) > > 'studio_id' => string '210' (length=3) > > 'genres' => string '8|9|10|29|58|' (length=13) > > > > Notice that they're all the same values, although the keys seem right. I > > don't understand why that happens because > > foreach($this->tmp_results as $k => $v) > > Should be changing $v each iteration I'd think. > > > > Honestly, I am baffled as to why those unsets() make no difference. All I > > can think is that the garbage collector doesn't run. But then I had also > > tried to force gc() and that still made no difference. *sigh* > > > > I had some other cockamamie idea where I'd use the same tmp_results array > in > > a tricky way to avoid a second array. The concept being I'd add 1 million > > to the ['id'] (which we want as the new array key), then unset the > existing > > sequential key, then when all done, loop through and shift all the keys by > 1 > > million thereby they'd be the right index ID. So add one and unset one > > immediately after. Clever right? 'cept it too made no difference on > memory. > > Same thing is happening as above where the gc() isn't running or something > > is holding all that memory until the end. *sigh* > > > > Then I tried a different way using array_combine() and noticed something > > very disturbing. > > http://www.php.net/manual/en/function.array-combine.php > > > > > > private function _normalize_result_set() > > { > > if (!$this->tmp_results || count($this->tmp_results) < 1) > > return; > > > > $D_start_mem_usage = memory_get_usage(); > > foreach($this->tmp_results as $k => $v) > > { > > $id = $v['id']; > > $tmp_keys[] = $id; > > > > if ($v['genres']) > > { > > $g = explode('|', $v['genres']); > > $this->tmp_results[$k]['g'] = $g; //this > causes a > > massive spike in memory usage > > } > > } > > //var_dump($tmp_keys, $this->tmp_results); exit; > > echo "\nMEMORY USED BEFORE array_combine: > > ".number_format(memory_get_usage() - $D_start_mem_usage)." PEAK: > > (".number_format(memory_get_peak_usage(true)).")<br>\n"; > > $this->tmp_results = array_combine($tmp_keys, > > $this->tmp_results); > > echo "\nMEMORY USED FOR array_combine: > > ".number_format(memory_get_usage() - $D_start_mem_usage)." PEAK: > > (".number_format(memory_get_peak_usage(true)).")<br>\n"; > > var_dump($tmp_keys, $this->tmp_results); exit; > > } > > > > Just the simple act of adding that 'g' variable element to the array > causes > > a massive change in memory usage. WHAT THE F!? > > > > MEMORY USED BEFORE array_combine: 105,315,264 PEAK: (224,395,264) > > MEMORY USED FOR array_combine: 106,573,040 PEAK: (224,395,264) > > > > And taking out the > > $this->tmp_results[$k]['g'] = $g; > > > > Results in > > MEMORY USED BEFORE array_combine: 8,050,456 PEAK: (78,118,912) > > MEMORY USED FOR array_combine: 8,050,376 PEAK: (86,507,520) > > > > Just as a wild guess, I also added 'g' to my SQL so that PHP would already > > have a placeholder variable there in tmp_results, but that made no > > difference. And still used up nearly double the memory as above. > > SELECT DISTINCT `id`, sag.`genres`, 'g' FROM. > > > > > Are you sure that the data is what you expect? I've never used an > object to hold the results of a query, but I'm picturing that your > foreach may not be working at all unless an object as a result is > totally different than the type of query results I always process. > > You're taking each object of the query results and assigning the key and > value to vars. But aren't query results 'keyless'? So what are you > actually assigning to $k and $v? > > If I'm wrong and using an object is quite different, forget I said > anything. :) Positive the data is right. It all works. It's just the memory usage is the issue. I'm not assigning an object. I'm assigning an array to a property (array) IN an object ($this). The query itself is using mysql_fetch_array() and MYSQL_ASSOC http://php.net/manual/en/function.mysql-fetch-array.php Results are certainly NOT keyless as you wouldn't be able to use them otherwise. They are a sequential multi-dimensional array starting at [0] ... [x] with column names as the sub-array hash keys and column values as the corresponding hash value. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
