On Mon, Aug 5, 2024 at 6:42 PM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote: > > On Mon, Aug 5, 2024 at 4:24 PM Jason Wang <jasowang@xxxxxxxxxx> wrote: > > > > On Mon, Aug 5, 2024 at 4:21 PM Jason Wang <jasowang@xxxxxxxxxx> wrote: > > > > > > Barry said [1]: > > > > > > """ > > > mm doesn't support non-blockable __GFP_NOFAIL allocation. Because > > > __GFP_NOFAIL without direct reclamation may just result in a busy > > > loop within non-sleepable contexts. > > > ""“ > > > > > > Unfortuantely, we do that under read lock. A possible way to fix that > > > is to move the pages allocation out of the lock into the caller, but > > > having to allocate a huge number of pages and auxiliary page array > > > seems to be problematic as well per Tetsuon [2]: > > > > > > """ > > > You should implement proper error handling instead of using > > > __GFP_NOFAIL if count can become large. > > > """ > > > > > I think the problem is it's hard to do the error handling in > fops->release() currently. vduse_dev_dereg_umem() should be the same, it's very hard to allow it to fail. > > So can we temporarily hold the user page refcount, and release it when > vduse_dev_open()/vduse_domain_release() is executed. The kernel page > allocation and memcpy can be done in vduse_dev_open() which allows > some error handling. Just to make sure I understand this, the free is probably not the big issue but the allocation itself. And if we do the memcpy() in open(), it seems to be a subtle userspace noticeable change? (Or I don't get how copying in vduse_dev_open() can help here). > > > > So I choose another way, which does not release kernel bounce pages > > > when user tries to register usersapce bounce pages. Then we don't need > > > to do allocation in the path which is not expected to be fail (e.g in > > > the release). We pay this for more memory usage but further > > > optimizations could be done on top. > > > > > > [1] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8ZQ2saLnUdfOGOLg@xxxxxxxxxxxxxx/T/#m3caef86a66ea6318ef94f9976ddb3a0ccfe6fcf8 > > > [2] https://lore.kernel.org/all/CACGkMEtcOJAA96SF9B8m-nZ1X04-XZr+nq8ZQ2saLnUdfOGOLg@xxxxxxxxxxxxxx/T/#m7ad10eaba48ade5abf2d572f24e185d9fb146480 > > > > > > Fixes: 6c77ed22880d ("vduse: Support using userspace pages as bounce buffer") > > > Signed-off-by: Jason Wang <jasowang@xxxxxxxxxx> > > > --- > > > > Note for YongJi: > > > > I can only test it without usersapce bounce pages as neither DPDK nor > > libvduse users use that. Would you want to review and have a test for > > this? > > > > I can do some tests for it. Great. > > Thanks, > Yongji > Thanks
