On 2024-10-22 03:24, Bernd Schubert wrote: > > > On 10/21/24 22:57, David Wei wrote: >> On 2024-10-21 04:47, Bernd Schubert wrote: >>> Hi David, >>> >>> On 10/21/24 06:06, David Wei wrote: >>>> [You don't often get email from dw@xxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] >>>> >>>> On 2024-10-15 17:05, Bernd Schubert wrote: >>>> [...] >>>>> >>> >>> ... >>> >>>> Hi Bernd, I applied this patchset to io_uring-6.12 branch with some >>>> minor conflicts. I'm running the following command: >>>> >>>> $ sudo ./build/example/passthrough_hp -o allow_other --debug-fuse --nopassthrough \ >>>> --uring --uring-per-core-queue --uring-fg-depth=1 --uring-bg-depth=1 \ >>>> /home/vmuser/scratch/source /home/vmuser/scratch/dest >>>> FUSE library version: 3.17.0 >>>> Creating ring per-core-queue=1 sync-depth=1 async-depth=1 arglen=1052672 >>>> dev unique: 2, opcode: INIT (26), nodeid: 0, insize: 104, pid: 0 >>>> INIT: 7.40 >>>> flags=0x73fffffb >>>> max_readahead=0x00020000 >>>> INIT: 7.40 >>>> flags=0x4041f429 >>>> max_readahead=0x00020000 >>>> max_write=0x00100000 >>>> max_background=0 >>>> congestion_threshold=0 >>>> time_gran=1 >>>> unique: 2, success, outsize: 80 >>>> >>>> I created the source and dest folders which are both empty. >>>> >>>> I see the following in dmesg: >>>> >>>> [ 2453.197510] uring is disabled >>>> [ 2453.198525] uring is disabled >>>> [ 2453.198749] uring is disabled >>>> ... >>>> >>>> If I then try to list the directory /home/vmuser/scratch: >>>> >>>> $ ls -l /home/vmuser/scratch >>>> ls: cannot access 'dest': Software caused connection abort >>>> >>>> And passthrough_hp terminates. >>>> >>>> My kconfig: >>>> >>>> CONFIG_FUSE_FS=m >>>> CONFIG_FUSE_PASSTHROUGH=y >>>> CONFIG_FUSE_IO_URING=y >>>> >>>> I'll look into it next week but, do you see anything obviously wrong? >>> >>> >>> thanks for testing it! I just pushed a fix to my libfuse branches to >>> avoid the abort for -EOPNOTSUPP. It will gracefully fall back to >>> /dev/fuse IO now. >>> >>> Could you please use the rfcv4 branch, as the plain uring >>> branch will soon get incompatible updates for rfc5? >>> >>> https://github.com/bsbernd/libfuse/tree/uring-for-rfcv4 >>> >>> >>> The short answer to let you enable fuse-io-uring: >>> >>> echo 1 >/sys/module/fuse/parameters/enable_uring >>> >>> >>> (With that the "uring is disabled" should be fixed.) >> >> Thanks, using this branch fixed the issue and now I can see the dest >> folder mirroring that of the source folder. There are two issues I >> noticed: >> >> [63490.068211] ---[ end trace 0000000000000000 ]--- >> [64010.242963] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:330 >> [64010.243531] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 11057, name: fuse-ring-1 >> [64010.244092] preempt_count: 1, expected: 0 >> [64010.244346] RCU nest depth: 0, expected: 0 >> [64010.244599] 2 locks held by fuse-ring-1/11057: >> [64010.244886] #0: ffff888105db20a8 (&ctx->uring_lock){+.+.}-{3:3}, at: __do_sys_io_uring_enter+0x900/0xd80 >> [64010.245476] #1: ffff88810f941818 (&fc->lock){+.+.}-{2:2}, at: fuse_uring_cmd+0x83e/0x1890 [fuse] >> [64010.246031] CPU: 1 UID: 0 PID: 11057 Comm: fuse-ring-1 Tainted: G W 6.11.0-10089-g0d2090ccdbbe #2 >> [64010.246655] Tainted: [W]=WARN >> [64010.246853] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014 >> [64010.247542] Call Trace: >> [64010.247705] <TASK> >> [64010.247860] dump_stack_lvl+0xb0/0xd0 >> [64010.248090] __might_resched+0x2f8/0x510 >> [64010.248338] __kmalloc_cache_noprof+0x2aa/0x390 >> [64010.248614] ? lockdep_init_map_type+0x2cb/0x7b0 >> [64010.248923] ? fuse_uring_cmd+0xcc2/0x1890 [fuse] >> [64010.249215] fuse_uring_cmd+0xcc2/0x1890 [fuse] >> [64010.249506] io_uring_cmd+0x214/0x500 >> [64010.249745] io_issue_sqe+0x588/0x1810 >> [64010.249999] ? __pfx_io_issue_sqe+0x10/0x10 >> [64010.250254] ? io_alloc_async_data+0x88/0x120 >> [64010.250516] ? io_alloc_async_data+0x88/0x120 >> [64010.250811] ? io_uring_cmd_prep+0x2eb/0x9f0 >> [64010.251103] io_submit_sqes+0x796/0x1f80 >> [64010.251387] __do_sys_io_uring_enter+0x90a/0xd80 >> [64010.251696] ? do_user_addr_fault+0x26f/0xb60 >> [64010.251991] ? __pfx___do_sys_io_uring_enter+0x10/0x10 >> [64010.252333] ? __up_read+0x3ba/0x750 >> [64010.252565] ? __pfx___up_read+0x10/0x10 >> [64010.252868] do_syscall_64+0x68/0x140 >> [64010.253121] entry_SYSCALL_64_after_hwframe+0x76/0x7e >> [64010.253444] RIP: 0033:0x7f03a03fb7af >> [64010.253679] Code: 45 0f b6 90 d0 00 00 00 41 8b b8 cc 00 00 00 45 31 c0 41 b9 08 00 00 00 41 83 e2 01 41 c1 e2 04 41 09 c2 b8 aa 01 00 00 0f 05 <c3> a8 02 74 cc f0 48 83 0c 24 00 49 8b 40 20 8b 00 a8 01 74 bc b8 >> [64010.254801] RSP: 002b:00007f039f3ffd08 EFLAGS: 00000246 ORIG_RAX: 00000000000001aa >> [64010.255261] RAX: ffffffffffffffda RBX: 0000561ab7c1ced0 RCX: 00007f03a03fb7af >> [64010.255695] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000009 >> [64010.256127] RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000008 >> [64010.256556] R10: 0000000000000000 R11: 0000000000000246 R12: 0000561ab7c1d7a8 >> [64010.256990] R13: 0000561ab7c1da00 R14: 0000561ab7c1d520 R15: 0000000000000001 >> [64010.257442] </TASK> > > Regarding issue one, does this patch solve it? > > diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c > index e518d4379aa1..304919bc12fb 100644 > --- a/fs/fuse/dev_uring.c > +++ b/fs/fuse/dev_uring.c > @@ -168,6 +168,12 @@ static struct fuse_ring_queue *fuse_uring_create_queue(struct fuse_ring *ring, > queue = kzalloc(sizeof(*queue), GFP_KERNEL_ACCOUNT); > if (!queue) > return ERR_PTR(-ENOMEM); > + pq = kcalloc(FUSE_PQ_HASH_SIZE, sizeof(struct list_head), GFP_KERNEL); > + if (!pq) { > + kfree(queue); > + return ERR_PTR(-ENOMEM); > + } > + > spin_lock(&fc->lock); > if (ring->queues[qid]) { > spin_unlock(&fc->lock); > @@ -186,11 +192,6 @@ static struct fuse_ring_queue *fuse_uring_create_queue(struct fuse_ring *ring, > INIT_LIST_HEAD(&queue->ent_in_userspace); > INIT_LIST_HEAD(&queue->fuse_req_queue); > > - pq = kcalloc(FUSE_PQ_HASH_SIZE, sizeof(struct list_head), GFP_KERNEL); > - if (!pq) { > - kfree(queue); > - return ERR_PTR(-ENOMEM); > - } > queue->fpq.processing = pq; > fuse_pqueue_init(&queue->fpq); > > > I think we don't need GFP_ATOMIC, but can do allocations before taking > the lock. This pq allocation is new in v4 and I forgot to put it into > the right place and it slipped through my very basic testing (I'm > concentrating on the design changes for now - testing will come back > with v6). Thanks, this patch fixed it for me. > >> >> If I am already in dest when I do the mount using passthrough_hp and >> then e.g. ls, it hangs indefinitely even if I kill passthrough_hp. > > I'm going to check in a bit. I hope it is not a recursion issue. > > > Thanks, > Bernd
