On Wed, Aug 7, 2024 at 2:52 PM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote: > > On Mon, Aug 5, 2024 at 4:21 PM Jason Wang <jasowang@xxxxxxxxxx> wrote: > > > > Barry said [1]: > > > > """ > > mm doesn't support non-blockable __GFP_NOFAIL allocation. Because > > __GFP_NOFAIL without direct reclamation may just result in a busy > > loop within non-sleepable contexts. > > ""“ > > > > Unfortuantely, we do that under read lock. A possible way to fix that > > is to move the pages allocation out of the lock into the caller, but > > having to allocate a huge number of pages and auxiliary page array > > seems to be problematic as well per Tetsuon [2]: > > > > """ > > You should implement proper error handling instead of using > > __GFP_NOFAIL if count can become large. > > """ > > > > So I choose another way, which does not release kernel bounce pages > > when user tries to register usersapce bounce pages. Then we don't need > > to do allocation in the path which is not expected to be fail (e.g in > > the release). We pay this for more memory usage but further > > optimizations could be done on top. > > > > [1] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8ZQ2saLnUdfOGOLg@xxxxxxxxxxxxxx/T/#m3caef86a66ea6318ef94f9976ddb3a0ccfe6fcf8 > > [2] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8ZQ2saLnUdfOGOLg@xxxxxxxxxxxxxx/T/#m7ad10eaba48ade5abf2d572f24e185d9fb146480 > > > > Fixes: 6c77ed22880d ("vduse: Support using userspace pages as bounce buffer") > > Signed-off-by: Jason Wang <jasowang@xxxxxxxxxx> > > --- > > Reviewed-by: Xie Yongji <xieyongji@xxxxxxxxxxxxx> > Tested-by: Xie Yongji <xieyongji@xxxxxxxxxxxxx> Thanks. > > Have tested it with qemu-storage-daemon [1]: > > $ qemu-storage-daemon \ > --chardev socket,id=charmonitor,path=/tmp/qmp.sock,server=on,wait=off \ > --monitor chardev=charmonitor \ > --blockdev driver=host_device,cache.direct=on,aio=native,filename=/dev/nullb0,node-name=disk0 > \ > --export type=vduse-blk,id=vduse-test,name=vduse-test,node-name=disk0,writable=on > > [1] https://github.com/bytedance/qemu/tree/vduse-umem Great, would you want to post them to the Qemu? > > > drivers/vdpa/vdpa_user/iova_domain.c | 18 ++++++++++-------- > > drivers/vdpa/vdpa_user/iova_domain.h | 1 + > > 2 files changed, 11 insertions(+), 8 deletions(-) > > > > diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c > > index 791d38d6284c..933d2f7cd49a 100644 > > --- a/drivers/vdpa/vdpa_user/iova_domain.c > > +++ b/drivers/vdpa/vdpa_user/iova_domain.c > > @@ -162,6 +162,7 @@ static void vduse_domain_bounce(struct vduse_iova_domain *domain, > > enum dma_data_direction dir) > > { > > struct vduse_bounce_map *map; > > + struct page *page; > > unsigned int offset; > > void *addr; > > size_t sz; > > @@ -178,7 +179,10 @@ static void vduse_domain_bounce(struct vduse_iova_domain *domain, > > map->orig_phys == INVALID_PHYS_ADDR)) > > return; > > > > - addr = kmap_local_page(map->bounce_page); > > + page = domain->user_bounce_pages ? > > + map->user_bounce_page : map->bounce_page; > > + > > + addr = kmap_local_page(page); > > do_bounce(map->orig_phys + offset, addr + offset, sz, dir); > > kunmap_local(addr); > > size -= sz; > > @@ -270,9 +274,8 @@ int vduse_domain_add_user_bounce_pages(struct vduse_iova_domain *domain, > > memcpy_to_page(pages[i], 0, > > page_address(map->bounce_page), > > PAGE_SIZE); > > - __free_page(map->bounce_page); > > } > > - map->bounce_page = pages[i]; > > + map->user_bounce_page = pages[i]; > > get_page(pages[i]); > > } > > domain->user_bounce_pages = true; > > @@ -297,17 +300,16 @@ void vduse_domain_remove_user_bounce_pages(struct vduse_iova_domain *domain) > > struct page *page = NULL; > > > > map = &domain->bounce_maps[i]; > > - if (WARN_ON(!map->bounce_page)) > > + if (WARN_ON(!map->user_bounce_page)) > > continue; > > > > /* Copy user page to kernel page if it's in use */ > > if (map->orig_phys != INVALID_PHYS_ADDR) { > > - page = alloc_page(GFP_ATOMIC | __GFP_NOFAIL); > > + page = map->bounce_page; > > memcpy_from_page(page_address(page), > > - map->bounce_page, 0, PAGE_SIZE); > > + map->user_bounce_page, 0, PAGE_SIZE); > > } > > - put_page(map->bounce_page); > > - map->bounce_page = page; > > + put_page(map->user_bounce_page); > > map->user_bounce_page = NULL? For safety, it might be better. Let me add this and post a V2/ Thanks > > Thanks, > Yongji >
