On Wed, Nov 20, 2024 at 02:58:54PM -0700, Brian Johannesmeyer wrote:
> These results are consistent across multiple runs. It seems that with
> DMAPOOL_DEBUG disabled, the patches introduce a significant
> performance hit. Let me know if you have any suggestions or further
> tests you'd like me to run.
That's what I was afraid of. I was working on the dma pool because it
showed significant lock contention on the pool for storage heavy
workloads, so cutting down the critical section was priority. With the
current kernel, the dma pool doesn't even register on the profiles
anymore, so it'd be great to keep it that way.
The idea for embedding the links in freed blocks was assuming a driver
wouldn't ask the kernel to free a dma block if the mapped device was
still using it. Untrustworthy hardware is why we can't have nice
things...
Here's my quick thoughts at this late hour, though I might have
something else tomorrow. If the links are external to the dma addr
being freed, then I think you need to change the entire alloc/free API
to replace the dma_addr_t handle with a new type, like a tuple of
{ dma_addr_t, priv_list_link }
That should make it possible to preserve the low constant time to alloc
and free in the critical section, which I think is a good thing to have.
I found 170 uses of dma_pool_alloc, and 360 dma_pool_free in the kernel,
so changing the API is no small task. :(
