I think I've identified the bulk of the memory "leak" I've been tracking down for the past few days. As it turns out, it doesn't seem to be a leak, but a problem with the SSL certificate caching. The certificate cache is set by dynamic_cert_mem_cache_size and defaults to 4MB. Squid assumes an SSL context is 1KB, so we can cache up to 4096 certificates: /// TODO: Replace on real size. #define SSL_CTX_SIZE 1024 Unfortunately the assumed size isn't even close - it looks like an SSL context usually weighs in at about 900KB! So the default limit means the cache can actually grow to around 3.6GB. To make matters worse, each worker gets its own cache, so in an 8-way SMP configuration, the default "4MB" cache size limit actually ends up as around 30GB. Possible fixes: 1. Stick with a static SSL_CTX_SIZE but use a more accurate estimate (I suggest 1MB / context, based on my observations). 2. Calculate the context sizes correctly. 3. In the config, specify the maximum number of certificates to be cached, rather than their total size. In any case, as it stands the defaults are pretty much guaranteed to cause a server to run out of memory. How "heavy weight" is SSL context generation anyway? i.e. if we expect to generate 20 certificates a minute (which is what I see on a production system in the first 5 minutes after startup), is that going to place a significant load on the system, or is that ok? the 900KB context size that I've observed seems pretty big to me. Does it sound reasonable, or is there some extra data being cached along with the context that could have been freed earlier? The instrumentation isn't good enough to be able to spot where this memory usage originates without extensive debugging. The memory isn't allocated to memory pools, so mgr:mem doesn't show it, and mgr:cached_ssl_cert relies on the incorrect estimate, so appears small and inconsequential. The processes can clearly be seen to grow rapidly, but it all just shows up as unaccounted memory in mgr:info. Secondly, mgr:cached_ssl_cert doesn't appear to work with SMP workers. My Squid process sizes are still larger than I would expect, so I'll need to do more investigation, but reducing dynamic_cert_mem_cache_size has stopped the rapid unbounded growth I have been seeing. -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com Direct contacts: Instant messager: xmpp:steve@xxxxxxxxxxxx Email: steve@xxxxxxxxxxxx Phone: sip:steve@xxxxxxxxxxxx Sales / enquiries contacts: Email: sales@xxxxxxxxxxxx Phone: +44-1792-825748 / sip:sales@xxxxxxxxxxxx Support contacts: Email: support@xxxxxxxxxxxx Phone: +44-1792-824568 / sip:support@xxxxxxxxxxxx _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
