On 2023/11/16 1:44, Mina Almasry wrote: > On Wed, Nov 15, 2023 at 1:21 AM Yunsheng Lin <linyunsheng@xxxxxxxxxx> wrote: >> >> On 2023/11/14 21:16, Jason Gunthorpe wrote: >>> On Tue, Nov 14, 2023 at 04:21:26AM -0800, Mina Almasry wrote: >>> >>>> Actually because you put the 'strtuct page for devmem' in >>>> skb->bv_frag, the net stack will grab the 'struct page' for devmem >>>> using skb_frag_page() then call things like page_address(), kmap, >>>> get_page, put_page, etc, etc, etc. >>> >>> Yikes, please no. If net has its own struct page look alike it has to >>> stay entirely inside net. A non-mm owned struct page should not be >>> passed into mm calls. It is just way too hacky to be seriously >>> considered :( >> >> Yes, that is something this patchset is trying to do, defining its own >> struct page look alike for page pool to support devmem. >> >> struct page for devmem will not be called into the mm subsystem, so most >> of the mm calls is avoided by calling into the devmem memory provider' >> ops instead of calling mm calls. >> >> As far as I see for now, only page_ref_count(), page_is_pfmemalloc() and >> PageTail() is called for devmem page, which should be easy to ensure that >> those call for devmem page is consistent with the struct page owned by mm. > > I'm not sure this is true. These 3 calls are just the calls you're > aware of. In your proposal you're casting mirror pages into page* and > releasing them into the net stack. You need to scrub the entire net > stack for mm calls, i.e. all driver code and all skb_frag_page() call > sites. Of the top of my head, the driver is probably calling > page_address() and illegal_highdma() is calling PageHighMem(). TCP > zerocopy receive is calling vm_insert_pages(). For net stack part, I believe your patch below is handling to aovid those mm calls? I don't include it in this patchset as I thought it is obvious that whatever the proposal is, we always need those checking. Maybe we should have included it to avoid this kind of confusion. https://lore.kernel.org/all/20231106024413.2801438-10-almasrymina@xxxxxxxxxx/ For driver part, I was thinking if the driver supports devmem, it should check that if it can call page_address() related call on a specific 'stuct page', or maybe we should introduce a new helper to make it obvious?
