On Mon, 17 Oct 2005, seth vidal wrote:
> On Mon, 2005-10-17 at 22:14 -0400, Konstantin Ryabitsev wrote:
>> On Mon, 2005-17-10 at 22:08 -0400, seth vidal wrote:
>>> where's it growing?
>>>
>>> it should just be reading the data in from the sqlite databases - not a
>>> huge memory hit, I don't believe.
>>
>> I haven't had a chance to take a long look at it, but it's like Rob
>> described -- it lists the same set of files over and over again,
>> especially from debuginfo packages, at least that's the impression I
>> had. It's hard to tell, since it scrolls really fast, but the kernel
>> version in the path was the same.
>>
>> I can try it again, but I'd rather not do this on my poor laptop, since
>> it gets very unhappy about this. :)
>>
>
> ugh - I wonder if it is actually coming from the result list getting too
> huge.
>
> wouldn't that just be ridiculous.
It has the feel of some sort of memory leak -- an iterated list that
causes python to constantly reallocate new memory without clearing the
previous reference or generating a new reference where it SHOULD be
reusing an old reference. I encountered some similar things in perl
once or twice -- this sort of interpreted language that provides
freeform data types that are opaque references to some sort of complex
underlying dynamically allocated struct can create some very odd bugs.
Not even "bugs", per se, legal code. But legal code that just destroys
your system with an unexpected side effect because of the way the
scripting langages reallocates growing lists. Sometimes this sort of
"bug" depends on >>how<< (in what order) something is referenced in the
code, not even on the size overall of the reference.
It could also be caused by nested loops in high dimensional searches
that don't avoid redoing already discovered permutations. It "should"
return a providing package a single time (but maybe show you all the
hits, also one time, in which you find the string within the package).
It "should" only store a single copy of this referent and uniq against
some sort of toplevel key (like the package name) before presenting at
all. It almost looks like it is generating the entire list of packages,
and the entire list of hits per package, per hit per package in the
entire list. If that makes sense. Instead of N (packages) * M (hits
per package, on average) lines of memory, N^M * N^M lines of memory...
or worse (as there may be more than these two dimensions, and N^M^O gets
big REAL fast...;-).
rgb
>
> -sv
>
>
> _______________________________________________
> Yum mailing list
> Yum@xxxxxxxxxxxxxxxxxxxx
> https://lists.dulug.duke.edu/mailman/listinfo/yum
>
--
Robert G. Brown http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@xxxxxxxxxxxx
