Hi David,
On 10/21/24 06:06, David Wei wrote:
[You don't often get email from dw@xxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
On 2024-10-15 17:05, Bernd Schubert wrote:
[...]
...
Hi Bernd, I applied this patchset to io_uring-6.12 branch with some
minor conflicts. I'm running the following command:
$ sudo ./build/example/passthrough_hp -o allow_other --debug-fuse --nopassthrough \
--uring --uring-per-core-queue --uring-fg-depth=1 --uring-bg-depth=1 \
/home/vmuser/scratch/source /home/vmuser/scratch/dest
FUSE library version: 3.17.0
Creating ring per-core-queue=1 sync-depth=1 async-depth=1 arglen=1052672
dev unique: 2, opcode: INIT (26), nodeid: 0, insize: 104, pid: 0
INIT: 7.40
flags=0x73fffffb
max_readahead=0x00020000
INIT: 7.40
flags=0x4041f429
max_readahead=0x00020000
max_write=0x00100000
max_background=0
congestion_threshold=0
time_gran=1
unique: 2, success, outsize: 80
I created the source and dest folders which are both empty.
I see the following in dmesg:
[ 2453.197510] uring is disabled
[ 2453.198525] uring is disabled
[ 2453.198749] uring is disabled
...
If I then try to list the directory /home/vmuser/scratch:
$ ls -l /home/vmuser/scratch
ls: cannot access 'dest': Software caused connection abort
And passthrough_hp terminates.
My kconfig:
CONFIG_FUSE_FS=m
CONFIG_FUSE_PASSTHROUGH=y
CONFIG_FUSE_IO_URING=y
I'll look into it next week but, do you see anything obviously wrong?
thanks for testing it! I just pushed a fix to my libfuse branches to
avoid the abort for -EOPNOTSUPP. It will gracefully fall back to
/dev/fuse IO now.
Could you please use the rfcv4 branch, as the plain uring
branch will soon get incompatible updates for rfc5?
https://github.com/bsbernd/libfuse/tree/uring-for-rfcv4
The short answer to let you enable fuse-io-uring:
echo 1 >/sys/module/fuse/parameters/enable_uring
(With that the "uring is disabled" should be fixed.)
The long answer for Miklos and others
IOCTL removal introduced a design issue, as now fuse-client
(kernel) does not know if fuse-server/libfuse wants to set
up io-uring communication.
It is not even possible to forbid FUSE_URING_REQ_FETCH after
FUSE_INIT reply, as io-uring is async. What happens is that
fuse-client (kernel) receives all FUSE_URING_REQ_FETCH commands
only after FUSE_INIT reply. And that although FUSE_URING_REQ_FETCH
is send out from libuse *before* replying to FUSE_INIT.
I had also added a comment for that into the code.
And the other issue is that libfuse now does not know if kernel supports
fuse-io-uring. That has some implications
- libfuse cannot write at start up time a clear error message like
"Kernel does not support fuse-over-io-uring, falling back to /dev/fuse IO"
- In the fallback code path one might want to adjust number of libfuse
/dev/fuse threads if io-uring is not supported - with io-uring typically
one thread might be sufficient - to handle FUSE_INTERRUPT.
My suggestion is that we introduce the new FUSE_URING_REQ_REGISTER (or
replace FUSE_URING_REQ_FETCH with that) and then wait in fuse-server
for completion of that command before sending out FUSE_URING_REQ_FETCH.
Thanks,
Bernd
