On Fri, 08 Mar, 2024 19:43:57 +0000 Haiyang Zhang <haiyangz@xxxxxxxxxxxxx> wrote: >> -----Original Message----- >> From: Jakub Kicinski <kuba@xxxxxxxxxx> >> Sent: Friday, March 8, 2024 2:23 PM >> To: Haiyang Zhang <haiyangz@xxxxxxxxxxxxx> >> Cc: Shradha Gupta <shradhagupta@xxxxxxxxxxxxxxxxxxx>; Shradha Gupta >> <shradhagupta@xxxxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux- >> hyperv@xxxxxxxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; >> netdev@xxxxxxxxxxxxxxx; Eric Dumazet <edumazet@xxxxxxxxxx>; Paolo Abeni >> <pabeni@xxxxxxxxxx>; Ajay Sharma <sharmaajay@xxxxxxxxxxxxx>; Leon >> Romanovsky <leon@xxxxxxxxxx>; Thomas Gleixner <tglx@xxxxxxxxxxxxx>; >> Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>; KY Srinivasan >> <kys@xxxxxxxxxxxxx>; Wei Liu <wei.liu@xxxxxxxxxx>; Dexuan Cui >> <decui@xxxxxxxxxxxxx>; Long Li <longli@xxxxxxxxxxxxx>; Michael Kelley >> <mikelley@xxxxxxxxxxxxx> >> Subject: Re: [PATCH] net :mana : Add per-cpu stats for MANA device >> >> On Fri, 8 Mar 2024 18:51:58 +0000 Haiyang Zhang wrote: >> > > Dynamic is a bit of an exaggeration, right? On a well-configured >> system >> > > each CPU should use a single queue assigned thru XPS. And for manual >> > > debug bpftrace should serve the purpose quite well. >> > >> > Some programs, like irqbalancer can dynamically change the CPU >> affinity, >> > so we want to add the per-CPU counters for better understanding of the >> CPU >> > usage. >> >> Do you have experimental data showing this making a difference >> in production? > Shradha, could you please add some data before / after enabling irqbalancer > which changes cpu affinity? > >> >> Seems unlikely, but if it does work we should enable it for all >> devices, no driver by driver. > There are some existing drivers, like mlx, rmnet, netvsc, etc. using percpu > counters. Are you suggesting we add a common API for all drivers? Wanted to chime in with regards to mlx. You might be conflating per-cpu with per-queue. When we run ethtool -S, we present counters per netdev queue rather than per-cpu. The number of queues we instantiate is related to CPUs but it not always 1-1. Jakub just recently supported a proper interface for per-queue stats counters that we are interested in supporting. https://lore.kernel.org/netdev/20240222223629.158254-1-kuba@xxxxxxxxxx/ -- Thanks, Rahul Rameshbabu
