On 8/24/24 6:54 AM, Joanne Koong wrote:
> On Thu, Aug 22, 2024 at 7:17 PM Jingbo Xu <jefflexu@xxxxxxxxxxxxxxxxx> wrote:
>>
>> On 8/23/24 5:19 AM, Joanne Koong wrote:
>>> On Thu, Aug 22, 2024 at 12:06 AM Jingbo Xu <jefflexu@xxxxxxxxxxxxxxxxx> wrote:
>>>>
>>>>
>>>>
>>>> On 8/14/24 7:22 AM, Joanne Koong wrote:
>>>>> Introduce two new sysctls, "default_request_timeout" and
>>>>> "max_request_timeout". These control timeouts on replies by the
>>>>> server to kernel-issued fuse requests.
>>>>>
>>>>> "default_request_timeout" sets a timeout if no timeout is specified by
>>>>> the fuse server on mount. 0 (default) indicates no timeout should be enforced.
>>>>>
>>>>> "max_request_timeout" sets a maximum timeout for fuse requests. If the
>>>>> fuse server attempts to set a timeout greater than max_request_timeout,
>>>>> the system will default to max_request_timeout. Similarly, if the max
>>>>> default timeout is greater than the max request timeout, the system will
>>>>> default to the max request timeout. 0 (default) indicates no timeout should
>>>>> be enforced.
>>>>>
>>>>> $ sysctl -a | grep fuse
>>>>> fs.fuse.default_request_timeout = 0
>>>>> fs.fuse.max_request_timeout = 0
>>>>>
>>>>> $ echo 0x100000000 | sudo tee /proc/sys/fs/fuse/default_request_timeout
>>>>> tee: /proc/sys/fs/fuse/default_request_timeout: Invalid argument
>>>>>
>>>>> $ echo 0xFFFFFFFF | sudo tee /proc/sys/fs/fuse/default_request_timeout
>>>>> 0xFFFFFFFF
>>>>>
>>>>> $ sysctl -a | grep fuse
>>>>> fs.fuse.default_request_timeout = 4294967295
>>>>> fs.fuse.max_request_timeout = 0
>>>>>
>>>>> Signed-off-by: Joanne Koong <joannelkoong@xxxxxxxxx>
>>>>> Reviewed-by: Josef Bacik <josef@xxxxxxxxxxxxxx>
>>>>> Reviewed-by: Bernd Schubert <bschubert@xxxxxxx>
>>>>> ---
>>>>> Documentation/admin-guide/sysctl/fs.rst | 17 ++++++++++
>>>>> fs/fuse/Makefile | 2 +-
>>>>> fs/fuse/fuse_i.h | 16 ++++++++++
>>>>> fs/fuse/inode.c | 19 ++++++++++-
>>>>> fs/fuse/sysctl.c | 42 +++++++++++++++++++++++++
>>>>> 5 files changed, 94 insertions(+), 2 deletions(-)
>>>>> create mode 100644 fs/fuse/sysctl.c
>>>>>
>>>>> diff --git a/Documentation/admin-guide/sysctl/fs.rst b/Documentation/admin-guide/sysctl/fs.rst
>>>>> index 47499a1742bd..44fd495f69b4 100644
>>>>> --- a/Documentation/admin-guide/sysctl/fs.rst
>>>>> +++ b/Documentation/admin-guide/sysctl/fs.rst
>>>>> @@ -332,3 +332,20 @@ Each "watch" costs roughly 90 bytes on a 32-bit kernel, and roughly 160 bytes
>>>>> on a 64-bit one.
>>>>> The current default value for ``max_user_watches`` is 4% of the
>>>>> available low memory, divided by the "watch" cost in bytes.
>>>>> +
>>>>> +5. /proc/sys/fs/fuse - Configuration options for FUSE filesystems
>>>>> +=====================================================================
>>>>> +
>>>>> +This directory contains the following configuration options for FUSE
>>>>> +filesystems:
>>>>> +
>>>>> +``/proc/sys/fs/fuse/default_request_timeout`` is a read/write file for
>>>>> +setting/getting the default timeout (in seconds) for a fuse server to
>>>>> +reply to a kernel-issued request in the event where the server did not
>>>>> +specify a timeout at mount. 0 indicates no timeout.
>>>>> +
>>>>> +``/proc/sys/fs/fuse/max_request_timeout`` is a read/write file for
>>>>> +setting/getting the maximum timeout (in seconds) for a fuse server to
>>>>> +reply to a kernel-issued request. If the server attempts to set a
>>>>> +timeout greater than max_request_timeout, the system will use
>>>>> +max_request_timeout as the timeout. 0 indicates no timeout.
>>>>
>>>> "0 indicates no timeout"
>>>>
>>>> I think 0 max_request_timeout shall indicate that there's no explicit
>>>> maximum limitation for request_timeout.
>>>
>>> Hi Jingbo,
>>>
>>> Ah I see where the confusion in the wording is (eg that "0 indicates
>>> no timeout" could be interpreted to mean there is no timeout at all
>>> for the connection, rather than no timeout as the max limit). Thanks
>>> for pointing this out. I'll make this more explicit in v5. I'll change
>>> the wording above for the "default_request_timeout" case too.
>>>
>>>>
>>>>
>>>>> diff --git a/fs/fuse/Makefile b/fs/fuse/Makefile
>>>>> index 6e0228c6d0cb..cd4ef3e08ebf 100644
>>>>> --- a/fs/fuse/Makefile
>>>>> +++ b/fs/fuse/Makefile
>>>>> @@ -7,7 +7,7 @@ obj-$(CONFIG_FUSE_FS) += fuse.o
>>>>> obj-$(CONFIG_CUSE) += cuse.o
>>>>> obj-$(CONFIG_VIRTIO_FS) += virtiofs.o
>>>>>
>>>>> -fuse-y := dev.o dir.o file.o inode.o control.o xattr.o acl.o readdir.o ioctl.o
>>>>> +fuse-y := dev.o dir.o file.o inode.o control.o xattr.o acl.o readdir.o ioctl.o sysctl.o
>>>>> fuse-y += iomode.o
>>>>> fuse-$(CONFIG_FUSE_DAX) += dax.o
>>>>> fuse-$(CONFIG_FUSE_PASSTHROUGH) += passthrough.o
>>>>> diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
>>>>> index 0a2fa487a3bf..dae9977fa050 100644
>>>>> --- a/fs/fuse/fuse_i.h
>>>>> +++ b/fs/fuse/fuse_i.h
>>>>> @@ -47,6 +47,14 @@
>>>>> /** Number of dentries for each connection in the control filesystem */
>>>>> #define FUSE_CTL_NUM_DENTRIES 5
>>>>>
>>>>> +/*
>>>>> + * Default timeout (in seconds) for the server to reply to a request
>>>>> + * if no timeout was specified on mount
>>>>> + */
>>>>> +extern u32 fuse_default_req_timeout;
>>>>> +/** Max timeout (in seconds) for the server to reply to a request */
>>>>> +extern u32 fuse_max_req_timeout;
>>>>> +
>>>>> /** List of active connections */
>>>>> extern struct list_head fuse_conn_list;
>>>>>
>>>>> @@ -1486,4 +1494,12 @@ ssize_t fuse_passthrough_splice_write(struct pipe_inode_info *pipe,
>>>>> size_t len, unsigned int flags);
>>>>> ssize_t fuse_passthrough_mmap(struct file *file, struct vm_area_struct *vma);
>>>>>
>>>>> +#ifdef CONFIG_SYSCTL
>>>>> +int fuse_sysctl_register(void);
>>>>> +void fuse_sysctl_unregister(void);
>>>>> +#else
>>>>> +static inline int fuse_sysctl_register(void) { return 0; }
>>>>> +static inline void fuse_sysctl_unregister(void) { return; }
>>>>> +#endif /* CONFIG_SYSCTL */
>>>>> +
>>>>> #endif /* _FS_FUSE_I_H */
>>>>> diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
>>>>> index 9e69006fc026..cf333448f2d3 100644
>>>>> --- a/fs/fuse/inode.c
>>>>> +++ b/fs/fuse/inode.c
>>>>> @@ -35,6 +35,10 @@ DEFINE_MUTEX(fuse_mutex);
>>>>>
>>>>> static int set_global_limit(const char *val, const struct kernel_param *kp);
>>>>>
>>>>> +/* default is no timeout */
>>>>> +u32 fuse_default_req_timeout = 0;
>>>>> +u32 fuse_max_req_timeout = 0;
>>>>> +
>>>>> unsigned max_user_bgreq;
>>>>> module_param_call(max_user_bgreq, set_global_limit, param_get_uint,
>>>>> &max_user_bgreq, 0644);
>>>>> @@ -1678,6 +1682,7 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx)
>>>>> struct fuse_conn *fc = fm->fc;
>>>>> struct inode *root;
>>>>> struct dentry *root_dentry;
>>>>> + u32 req_timeout;
>>>>> int err;
>>>>>
>>>>> err = -EINVAL;
>>>>> @@ -1730,10 +1735,16 @@ int fuse_fill_super_common(struct super_block *sb, struct fuse_fs_context *ctx)
>>>>> fc->group_id = ctx->group_id;
>>>>> fc->legacy_opts_show = ctx->legacy_opts_show;
>>>>> fc->max_read = max_t(unsigned int, 4096, ctx->max_read);
>>>>> - fc->req_timeout = ctx->req_timeout * HZ;
>>>>> fc->destroy = ctx->destroy;
>>>>> fc->no_control = ctx->no_control;
>>>>> fc->no_force_umount = ctx->no_force_umount;
>>>>> + req_timeout = ctx->req_timeout ?: fuse_default_req_timeout;
>>>>> + if (!fuse_max_req_timeout)
>>>>> + fc->req_timeout = req_timeout * HZ;
>>>>> + else if (!req_timeout)
>>>>> + fc->req_timeout = fuse_max_req_timeout * HZ;
>>>>
>>>> So if fuse_max_req_timeout is non-zero and req_timeout is zero (either
>>>> because of 0 fuse_default_req_timeout, or explicit "-o request_timeout =
>>>> 0" mount option), the final request timeout is exactly
>>>> fuse_max_req_timeout, which is unexpected as I think 0
>>>> fuse_default_req_timeout, or "-o request_timeout=0" shall indicate no
>>>> timeout.
>>>
>>> fuse_max_req_timeout takes precedence over fuse_default_req_timeout
>>> (eg if the system administrator wants to enforce a max limit on fuse
>>> timeouts, that is imposed even if a specific fuse server didn't
>>> indicate a timeout or indicated no timeout). Sorry, that wasn't made
>>> clear in the documentation. I'll add that in for v5.
>>
>> OK that is quite confusing. If the system admin wants to enforce a
>> timeout, then a non-zero fuse_default_req_timeout is adequate. What's
>> the case where fuse_default_req_timeout must be 0, and the aystem admin
>> has to impose the enforced timeout through fuse_max_req_timeout?
>>
>> IMHO the semantics of fuse_max_req_timeout is not straightforward and
>> can be confusing if it implies an enforced timeout when no timeout is
>> specified, while at the same time it also imposes a maximum limitation
>> when timeout is specified.
>
> In my point of view, max_req_timeout is the ultimate safeguard the
> administrator can set to enforce a timeout on all fuse requests on the
> system (eg to mitigate rogue servers). When this is set, this
> guarantees that absolutely no request will take longer than
> max_req_timeout for the server to respond.
>
> My understanding of /proc/sys sysctls is that ACLs can be used to
> grant certain users/groups write permission for specific sysctl
> parameters. So if a user wants to enforce a default request timeout,
> they can set that. If that timeout is shorter than what the max
> request timeout has been set to, then the request should time out
> earlier according to that desired default timeout. But if it's greater
> than what the max request timeout allows, then the max request timeout
> limits the timeout on the request (the max request timeout is the
> absolute upper bound on how long a request reply can take). It doesn't
> matter if the user set no timeout as the default req timeout - what
> matters is that there is a max req timeout on the system, and that
> takes precedence for enforcing how long request replies can take.
>
Sorry for the late reply, just back from vacation these days.
Anyway, if max_req_timeout enforces a maximum timeout no matter whether
the fuse server explicitly specifies a timeout or not, then the
semantics of fuse_default_req_timeout seems a little bit overlapped with
max_req_timeout, right? The only place where fuse_default_req_timeout
plays a role is when ctx->req_timeout and fuse_max_req_timeout are both
zero, in which case we can get the same effect if we eliminate
fuse_default_req_timeout and configure a non-zero fuse_max_req_timeout.
--
Thanks,
Jingbo
