Jeff King <peff@xxxxxxxx> wrote: > On Wed, Jun 30, 2021 at 12:01:32AM +0000, Eric Wong wrote: > > > This series is now down to a single patch. > > > > I wanted to make things more transparent to users without > > privileges to raise sys.vm.max_map_count and/or RLIMIT_DATA; > > but it doesn't seem possible to account for libc/zlib/etc. doing > > mmap() without our knowledge (usually via malloc). > > Oh, I should have read this one before reviewing the inner parts of v1. :) > > In general I agree that trying to manage our map count can never be > foolproof. As you note, other parts of the system may contribute to that > count. But even within Git, we don't have any mechanism for unmapping > many non-packfiles. So if you have 30,000 packs, you may hit the limit > purely via the .idx files (and ditto for the new .rev files, and > probably commit-graph files, etc). Yeah, the most annoying thing with my original series was when I hit "inflate: out of memory" once I stopped xmmap from dying. I suspect that would be a worse far error message for users who aren't familiar with how malloc works. > That said, I'm not opposed to handling xmmap() failures more gracefully, > as your series did. It's not foolproof, but it might help in some cases. I've also been wondering if we can maintain a watermark based on reading the contents of /proc/sys/vm/max_map_count and the mappings we make. Then we could start dropping mappings when we hit half (or some other threshold) of that sysctl. Similar for RLIMIT_DATA (though that defaults to unlimited on my Debian system). OTOH, I also wonder if we're overusing mmap when we could be just as well served with pread. I'm not up-to-date on modern mmap performance and maybe CPU vulnerability mitigations nowadays make mmap more compelling. However, once upon a time in 2006, pread could be a hair quicker: https://lore.kernel.org/git/Pine.LNX.4.64.0612182234260.3479@xxxxxxxxxxxxxx/T/#u (But that info could be out-of-date...) I'm also somewhat paranoid when it comes to mmap since rogue processes could be truncating mmap-ed files to cause bus errors. > > So I think giving users some information to feed their sysadmins > > is the best we can do in this situation: > > This seems OK to me, too. Translators might complain a bit about the > message-lego. I don't have a strong opinion. *shrug* I saw my original patches already ended up in `seen' (commit 7b79212a93c375365c06cab5c0018ab97a4185cf)
