On Sun, Aug 11, 2019 at 12:08:53PM -0700, Carlo Arenas wrote: > there is also the risk that xmalloc might not be sufficiently thread > safe (ex: when it triggers unuse_one_window() through the use of a > try_to_free_routine in packfile.c but we could mitigate the risk for > now by doing it only first #ifdef NED (I promise to take it out before > it even gets to next, assuming Junio would trust me enough after the > last blow up to even get it into pu) Yes, our xmalloc() is very much not thread-safe, and I suspect multithreaded bits that don't override try_to_free_routine, like git-grep, are probably buggy right now (assuming they xmalloc() outside of the big global lock). This has been the case for ages (see [1] from 2011, for example) and I don't think anybody has ever complained. Which leads me to believe that try_to_free_routine() rarely actually kicks in. I've long suspected that the whole try_to_free_pack_memory thing is more trouble than it's worth these days. We're dealing with mmap'd files, so the OS is going to do a much better job at responding to memory pressure by dropping individual pages. So the only thing we're really freeing is address space. On 64-bit systems, we have plenty of that to go around. On 32-bit systems, it might. But we introduced it[2] at the same time as sliding windows[3], so it's not clear how much it helps (keep in mind that even without this we'd still free old pack windows to make room for more mmap'd packs). And as a cost, xmalloc() isn't thread safe, and is a lot harder to reason about (e.g., something simple like strbuf_grow() might trigger quite a bit more work than you'd expect). I think it might be worth just eliminating the whole idea. [1] https://public-inbox.org/git/20110831015936.GB2519@xxxxxxxxxxxxxxxxxxxxx/ [2] 97bfeb34df (Release pack windows before reporting out of memory., 2006-12-24) [3] 60bb8b1453 (Fully activate the sliding window pack access., 2006-12-23) -Peff
