On Tue, 2024-05-28 at 13:59 +0200, Miklos Szeredi wrote: > External email: Use caution opening links or attachments > > > On Tue, 28 May 2024 at 10:52, Jingbo Xu <jefflexu@xxxxxxxxxxxxxxxxx> > wrote: > > > > Hi Peter-Jan, > > > > Thanks for the amazing work. > > > > I'd just like to know if you have any plan of making fiq and fiq- > > >lock > > more scalable, e.g. make fiq a per-CPU software queue? We have also identified the single fiq and fiq->lock as a logically large bottleneck in our system's data path. But we don't have hard proof for this just yet. I also still need to investigate what the impact of having multiple locks would be on the file system consistency in the FUSE driver. File systems like their locks... Can anyone here comment on that? > > Doing a per-CPU queue is not necessarily a good idea: async requests > could overload one queue while others are idle. > > One idea is to allow request to go through a per-CPU fast path if the > respective listener is idle. Otherwise the request would enter the > default slow queue, where idle listeners would pick requests (like > they do now). > > Might make sense to also optimize the way this picking is done if > there are several idle listeners according to how close the listener's > CPU is to the one that generated the request. > > Thanks, > Miklos I am not that familiar yet with the FUSE driver. What are these listeners you are referring to? If I follow a FUSE file system operation like fuse_statfs, I see a single-threaded path towards the wake_pending_and_unlock of virtio-fs via fuse_simple_request. Or are these listeners something from the virtiofsd implementation? I think the optimality of another "simple" multi-queue layer but in the FUSE driver would depend on the virtio-fs device implementation (busy polling or thread pooling like virtiofsd) and how the file system operates (asynchronously over the network or locally accessing the disk). Are you aware of other device drivers that perform smart tricks like checking if the per-core queue is full and using another empty one?
