-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 1/10/2014 3:59 a.m., Steve Hill wrote: > > On 30.09.14 15:13, Amos Jeffries wrote: > >> IIRC the valgrind report is strangely at the end of mgr:info >> rather than mgr:mem. > > In that case the wiki is wrong. :) Anyway, it doesn't show up on > either of them for me so I guess you might be right that it isn't > "real" leaked memory. (I still thought I'd see some indication > that it was invoking the valgrind report though, even if it > contained no data) > >> With one small exception all the reports of memory "leaks" in >> 3.2/3.3/3.4 series are in fact memory over-allocations. The >> difference is that over-allocated memory is still referenced by >> some part of Squid, so it *does not* show up in a leak finder >> like valgrind. > > Do you have any advice for finding out what memory is still > referenced in large amounts? Since this is unaccounted memory, it > presumably doesn't appear in mgr:mem (at least, I can't see > anything obvious in there)? > > I'm trying to figure out if there's a way of convincing valgrind to > dump info about all the currently allocated memory while the > program is still running - there would be a lot of legitimate stuff > in the report, but hopefully a few hundred MB of memory that > shouldn't be there would stick out like a sore thumb. That would be lovely. If you can find it I'd like to know too :-) A third option is using a ALL,9 cache.log tracer. We have begun instrumenting class contstructors and destructors with debug statements to record their lifetimes regardless of whether they are accounted. A lot are not yet done though. The bzr repo contains a debug script at scripts/find-alive.pl which correlates constructors to destructors in that log. It is useful for scanning a cache.log while Squid is still running, crashed, or paused in debugger to show what Jobs/requests etc are currently active. > >> If you are using a 64-bit OS then the unaccounted numbers there >> are bogus. They are based on mallinfo() lookup results which >> suffer from 32-bit wrap issues. Use the OS command line to get a >> memory report from top or similar instead. > > In this case I don't believe the data is bogus. I have 8 workers, > which top shows as: 26249 squid 20 0 871m 782m 5348 S 0.7 > 10.0 8:44.81 squid 26245 squid 20 0 732m 644m 5344 S 0.3 > 8.3 8:00.77 squid 26244 squid 20 0 706m 617m 5348 S 1.0 > 7.9 7:42.29 squid 26250 squid 20 0 699m 613m 5348 S 3.6 > 7.9 4:43.49 squid 26246 squid 20 0 699m 612m 5348 S 2.3 > 7.9 6:12.78 squid 26251 squid 20 0 662m 576m 5348 S 0.7 > 7.4 5:45.11 squid 26248 squid 20 0 649m 564m 5348 S 2.3 > 7.2 9:22.91 squid 26247 squid 20 0 603m 518m 5348 S 1.3 > 6.6 4:45.47 squid > > Adding the sizes in the "virt" column gives me 5621MB. The > combined RSS is 4926MB. Process 26250 has just 2MB in swap, the > rest have no swap at all. I guess the difference between the RSS > and Virt totals can probably be almost entirely accounted for by > the executables. > > mallinfo() says there is about 4897MB in the arena - close enough > to sum of the RSS that I'm inclined to believe it. Since no one > process exceeds 2GB, mallinfo() is probably trustworthy in this > case. > > The accounted for memory is 440MB - potentially higher than I > would expect for a Squid with caching disabled, but certainly low > enough to not be an immediate concern. But that gives us about > 4.4GB of memory unaccounted for (by subtracting 440MB from the sum > of the RSS as shown by top). > > 4.4GB of unaccounted memory after running for just 6 hours is a > significant amount, and if left to their own devices Squid > continues to eat all the memory, leaving the machine thrashing > swap. > > I count 531 sockets in netstat, so my guess is that it isn't just > data associated with some open sockets: netstat -apn|egrep > '26249|26245|26244|26250|26246|26251|26248|26247' -c 531 > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJUKsh9AAoJELJo5wb/XPRjUnAIALX5yfqY/XOVOcH90OXUree2 sHDJdCB5zlr4EO94qvichnkUm5bCMlg78dkPv8Nwyd43nG4SUUBNt4nh2dvZxYHn 8scHWFA1ZMsbl3k3dQ3eEHNCm+PfugvBo34ZokkMpSR8mLZhh9vqxXDhh9rwj6ez K8mKnYMbNI/z/DOHhJxcM1NdnbYIyO84EUQfb9RGaJaKb8LDNLt4j6yCEeKa2Z8N YcnPdA9VZZ3RCpImq0Py8U0kZbCPPBRs+cJAvUIgaAO080ZTrbF+bLZ+dHe4W7hA JMxXUD4lfEy2HVja2+jfKBcJQwVCDwBK0tcu76bQ64WIrfuIIc/NyIOBwEhPT1I= =cLQ3 -----END PGP SIGNATURE----- _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
