Hi,
While working on PostgreSQL [1] we've stumbled upon a question regarding
resizing of shared mappings without conflicting with any other possible
mappings. Before making any wrong conclusions, I would love to get some
consultation from kernel folks on that topic.
To put it into a context, PostgreSQL uses anonymous shared memory
mapping as a buffer cache for data. The mapping size is configured at
the start, and could not be changed without a restart. Now, we would
like to make it more flexible and allow to change it at runtime, ideally
without changing already used addresses and copying stuff back and
forth.
The idea is to place the shared mapping at a specified address (with
MAP_FIXED if needed) with a gap, then use mremap to resize it into the
gap. This approach has an open question -- how to make sure there will
be no other mapping created withing the same address space, where we
want to expand the shared mapping? E.g. the shared mapping was created,
then large memory allocation caused another mapping to be created close
to it, so that expanding is not possible.
One way to solve it would be to rely on the fact that the kernel, if
addr is NULL, always picks up a lowest address that allows for enough
space (AFAICT in vma_iter_area_lowest). This makes it possible to leave
the gap to lowest address large enough to accumulate any possible
allocations, as well as leaving space for resizing, something like:
01339000-0139e000 [heap]
0139e000-014aa000 [heap]
7f2dd72f6000-7f2dfbc9c000 /memfd:strategy (deleted)
7f2e0209c000-7f2e269b0000 /memfd:checkpoint (deleted)
7f2e2cdb0000-7f2e516b4000 /memfd:iocv (deleted)
7f2e57ab4000-7f2e7c478000 /memfd:descriptors (deleted)
7f2ebc478000-7f2ee8d3c000 /memfd:buffers (deleted)
^ note the distance between two mappings,
which is intended for resize
7f3168d3c000-7f318d600000 /memfd:main (deleted)
^ here is where the gap starts
7f4194c00000-7f4194e7d000
^ this one is an anonymous maping created due to large
memory allocation after shared mappings were created
7f4195000000-7f419527d000
7f41952dc000-7f4195416000
7f4195416000-7f4195600000 /dev/shm/PostgreSQL.2529797530
7f4195600000-7f41a311d000 /usr/lib/locale/locale-archive
7f41a317f000-7f41a3200000
7f41a3200000-7f41a3201000 /usr/lib64/libicudata.so.74.2
[...]
What I'm not sure about is how safe it is to rely on such behavior in
long term? It seems like the way how mapping address will be chosen if
addr is NULL is not standartized, and considered to be implementation
specific. What are the chances this behavior becomes different over
time?
Another question is whether there are better ways or mechanisms to
achieve resizable shared mapping? The topic sounds natural enough, it
feels like there should be other use cases that require this as well.
[1]: https://www.postgresql.org/message-id/scor5gscd42d4nwszuwvtwss6e22fg3dnvxmqwrcsdkpyyigny%40efjlkj6ccv7u
