On Thu, 29 Jun 2023 19:27:46 -0700 Mina Almasry wrote: > I've discussed your page pool frontend idea with our gve owners and > the idea is attractive. In particular it would be good not to insert > much custom code into the driver to support device memory pages or > other page types. I plan on trying to change my approach to match the > page pool provider you have in progress here: > https://github.com/kuba-moo/linux/tree/pp-providers > > In particular the main challenge right now seems to be that my device > memory pages are ZONE_DEVICE pages, which can't be inserted to the > page pool as-is due to the union in struct page between the page pool > entries and the ZONE_DEVICE entries. I have some ideas on how to work > around that I'm looking into. Right, have you talked to Willem? I mentioned to him that I initially pursued the idea of using the page pool as an abstraction but I realized that fitting both the dma addr and the frag reference count into struct page which isn't just a basic page will be a challenge. So the second best thing seems to be to create an API which matches the page pool API, and have it select between a page pool and another provider based on user configuration. As you said the main goal is to be able to feed kernel / user / device memory to the driver without having to modify driver code. I thought this would live "above" the page pool (perhaps that's what you mean by gen_pool) but Jesper brought up integrating it into the (middle of?) page pool and have page pool switch on the pp_magic. No strong preference on which on is better, seems like something that can only be ironed out by modifying a couple of drivers to find out what fits best :( I'm hoping to jump back into the pp_provider work and finish that up over the next few days, to at least post a PoC. Even if it doesn't work for user / device memory - being able to feed the page pool from huge pages is already a big win for the IOTLB.
