Hi Bernd and Miklos, On 6/3/24 11:19 PM, Miklos Szeredi wrote: > On Mon, 3 Jun 2024 at 16:43, Bernd Schubert <bernd.schubert@xxxxxxxxxxx> wrote: >> >> >> >> On 6/3/24 08:17, Jingbo Xu wrote: >>> Hi, Miklos, >>> >>> We spotted a performance bottleneck for FUSE writeback in which the >>> writeback kworker has consumed nearly 100% CPU, among which 40% CPU is >>> used for copy_page(). >>> >>> fuse_writepages_fill >>> alloc tmp_page >>> copy_highpage >>> >>> This is because of FUSE writeback design (see commit 3be5a52b30aa >>> ("fuse: support writable mmap")), which newly allocates a temp page for >>> each dirty page to be written back, copy content of dirty page to temp >>> page, and then write back the temp page instead. This special design is >>> intentional to avoid potential deadlocked due to buggy or even malicious >>> fuse user daemon. >> >> I also noticed that and I admin that I don't understand it yet. The commit says >> >> <quote> >> The basic problem is that there can be no guarantee about the time in which >> the userspace filesystem will complete a write. It may be buggy or even >> malicious, and fail to complete WRITE requests. We don't want unrelated parts >> of the system to grind to a halt in such cases. >> </quote> >> >> >> Timing - NFS/cifs/etc have the same issue? Even a local file system has no guarantees >> how fast storage is? > > I don't have the details but it boils down to the fact that the > allocation context provided by GFP_NOFS (PF_MEMALLOC_NOFS) cannot be > used by the unprivileged userspace server (and even if it could, > there's no guarantee, that it would). > > When this mechanism was introduced, the deadlock was a real > possibility. I'm not sure that it can still happen, but proving that > it cannot might be difficult. IIUC, there are two sources that may cause deadlock: 1) the fuse server needs memory allocation when processing FUSE_WRITE requests, which in turn triggers direct memory reclaim, and FUSE writeback then - deadlock here 2) a process that trigfgers direct memory reclaim or calls sync(2) may hang there forever, if the fuse server is buggyly or malicious and thus hang there when processing FUSE_WRITE requests Thus the temp page design was introduced to avoid the above potential issues. I think case 1 may be fixed (if any), but I don't know how case 2 can be avoided as any one could run a fuse server in unprivileged mode. Or if case 2 really matters? Please correct me if I miss something. -- Thanks, Jingbo