On Thu, Jan 23, 2025 at 1:51 AM Nick Renner <nr2185@xxxxxxx> wrote: Is this on-demand > allocation and freeing of pages necessary? What are some reasons that > the full pipe capacity shouldn't be allocated for the lifetime of a > pipe? > Keeping all the capacity constantly allocated is just memory waste. On my laptop alone there is 277 pipes which are empty vast majority of the time. This translates to over 17MB of kernel memory which is not allocated just in case. I'll note though there is a possible workaround: the BSD systems back pipes with pages which can be swapped out. I don't know if this is worth doing though. > Some of my results indicate that just this on-demand page management > contributes something like ~25% overhead when continuously writing and > reading multi-page buffers. But I'm not sure if there are some memory > management concerns that are deemed more important than the cost of > this overhead. The real problem is the fact that the allocator is dog slow. Before even considering not allocating these pages as needed, one would want to fix that. -- Mateusz Guzik <mjguzik gmail.com>
