On Tue 04-02-25 06:06:42, Christoph Hellwig wrote: > On Tue, Feb 04, 2025 at 01:50:08PM +1100, Dave Chinner wrote: > > I doubt that will create enough concurrency for a typical small > > server or desktop machine that only has a single NUMA node but has a > > couple of fast nvme SSDs in it. > > > > > 2) Fixed number of writeback contexts, say min(10, numcpu). > > > 3) NUMCPU/N number of writeback contexts. > > > > These don't take into account the concurrency available from > > the underlying filesystem or storage. > > > > That's the point I was making - CPU count has -zero- relationship to > > the concurrency the filesystem and/or storage provide the system. It > > is fundamentally incorrect to base decisions about IO concurrency on > > the number of CPU cores in the system. > > Yes. But as mention in my initial reply there is a use case for more > WB threads than fs writeback contexts, which is when the writeback > threads do CPU intensive work like compression. Being able to do that > from normal writeback threads vs forking out out to fs level threads > would really simply the btrfs code a lot. Not really interesting for > XFS right now of course. > > Or in other words: fs / device geometry really should be the main > driver, but if a file systems supports compression (or really expensive > data checksums) being able to scale up the numbes of threads per > context might still make sense. But that's really the advanced part, > we'll need to get the fs geometry aligned to work first. As I'm reading the thread it sounds to me the writeback subsystem should provide an API for the filesystem to configure number of writeback contexts which would be kind of similar to what we currently do for cgroup aware writeback? Currently we create writeback context per cgroup so now additionally we'll have some property like "inode writeback locality" that will also influence what inode->i_wb gets set to and hence where mark_inode_dirty() files inodes etc. Then additionally you'd like to have possibility to have more processes operate on one struct wb_writeback (again configurable by filesystem?) to handle cases of cpu intensive writeback submission more efficiently. Sounds about right? Honza -- Jan Kara <jack@xxxxxxxx> SUSE Labs, CR
