> On Mon, May 30, 2022 at 04:29:56PM +0200, nicolas saenz julienne wrote: > > On Mon, 2022-05-30 at 02:40 +0200, Frederic Weisbecker wrote: > > > On Sat, May 28, 2022 at 04:24:50PM +0200, Peter Zijlstra wrote: > > > > On Fri, May 27, 2022 at 10:30:18AM +0200, Juri Lelli wrote: > > > > > Hi, > > > > > > > > > > On 26/05/22 14:37, Tejun Heo wrote: > > > > > > On Thu, May 26, 2022 at 08:28:43PM -0400, Waiman Long wrote: > > > > > > > I am thinking along the line that it will not be hierarchical. However, > > > > > > > cpuset can be useful if we want to have multiple isolated partitions > > > > > > > underneath the top cpuset with different isolation attributes, but no more > > > > > > > sub-isolated partition with sub-attributes underneath them. IOW, we can only > > > > > > > set them at the first level under top_cpuset. Will that be useful? > > > > > > > > > > > > At that point, I'd just prefer to have it under /proc or /sys. > > > > > > > > > > FWIW, I was under the impression that this would nicely fit along the > > > > > side of other feaures towards implenting dynamic isolation of CPUs (say > > > > > https://lore.kernel.org/lkml/20220510153413.400020-1-longman@xxxxxxxxxx/ > > > > > for example). Wouldn't be awkward to have to poke different places to > > > > > achieve isolation at runtime? > > > > > > > > This, that's what I was thinking. > > > > > > > > My main objection to the whole thing is that it's an RCU_NOCB specific > > > > interface. *That* I think is daft. > > > > > > > > I was thinking a partition would be able to designate a house-keeping > > > > sub-partition/mask, but who cares about all the various different > > > > housekeeping parties. > > > > > > It's time for the isolation users to step up here! I very rarely hear from them > > > and I just can't figure out by myself all the variants of uses for each of the > > > isolation features. May be some people are only interested in nocb for some > > > specific uses, or may be it never makes sense without nohz full and all the rest > > > of the isolation features. So for now I take the very cautious path to split the > > > interface. > > > > OK, my 2 cents. I personally deal with virtualisation setups that involve RT > > and CPU isolation on both host and guests. > > > > The main use-case ATM is running DPDK-like workloads. We want to achieve > > latencies in the order of tens of microseconds, so it's essential to avoid > > entering the kernel at all cost. So, no HW interrupts, sched tick, RCU > > callbacks, clocksource watchdogs, softlockup, intel_pstate, timers, etc... > > Everything is deferred onto housekeeping CPUs or disabled. > > > > Then we have setups that need to deal with HW on the host, exposed to the guest > > through emulation or VirtIO. The same rules apply really, except for some IRQ > > affinity tweaks and sched priority magic. > > > > I find it hard to see how running RCU callback locally could be useful to any > > latency sensitive workload. > > > > Frederic, out of curiosity, do you have a use-case in mind that might benefit > > from nohz_full but not rcu_nocb? Maybe HPC? On Mon, May 30, 2022 at 8:42 AM Paul E. McKenney <paulmck@xxxxxxxxxx> wrote: > Would users looking for millisecond-scale latencies want rcu_nocbs but > not nohz_full, that is, the other way around? On Intel processors running 5.15 with the timersd patches from Siewior backported and Weisbecker's bug-fix, choosing CONFIG_HZ_PERIODIC prevents cores from entering deeper C-states when hrtimers are pending. With CONFIG_NO_HZ_COMMON, the cores do not service non-blocking timer callbacks until another thread wakes them. Since low latency is critical, this is a use case where NO_HZ_FULL will not work but rcu_nocbs is needed. -- Alison Chaiken, Aurora Innovation
