On Sun, Mar 15, 2020 at 01:45:05PM -0400, Mathieu Desnoyers wrote: > ----- On Mar 13, 2020, at 11:42 AM, paulmck paulmck@xxxxxxxxxx wrote: > > > On Fri, Mar 13, 2020 at 03:41:46PM +0100, Frederic Weisbecker wrote: > >> On Thu, Mar 12, 2020 at 11:16:18AM -0700, Paul E. McKenney wrote: > >> > Hello! > >> > > >> > This series provides two variants of Tasks RCU, a rude variant inspired > >> > by Steven Rostedt's use of schedule_on_each_cpu(), and a tracing variant > >> > requested by the BPF folks and perhaps also of use for other tracing > >> > use cases. > >> > > >> > The tracing variant has explicit read-side markers to permit finite grace > >> > periods even given in-kernel loops in PREEMPT=n builds It also protects > >> > code in the idle loop, on exception entry/exit paths, and on the various > >> > CPU-hotplug online/offline code paths, thus having protection properties > >> > similar to SRCU. However, unlike SRCU, this variant avoids expensive > >> > instructions in the read-side primitives, thus having read-side overhead > >> > similar to that of preemptible RCU. > >> > > >> > There are of course downsides. The grace-period code can send IPIs to > >> > CPUs, even when those CPUs are in the idle loop or in nohz_full userspace. > >> > It is necessary to scan the full tasklist, much as for Tasks RCU. There > >> > is a single callback queue guarded by a single lock, again, much as for > >> > Tasks RCU. If needed, these downsides can be at least partially remedied > >> > >> So what we trade to fix the issues we are having with tracing against extended > >> grace periods, we lose in CPU isolation. That worries me a bit as tracing can > >> be thoroughly used with nohz_full and CPU isolation. > > > > First, disturbing nohz_full CPUs can be avoided by the sysadm simply > > refusing to remove tracepoints while sensitive applications are running > > on nohz_full CPUs. > > I doubt this approach will survive real-life. Nothing survives real life, at least not indefinitely. ;-) > > Second, for non-CPU-bound real-time programs with mostly-idle CPUs, > > I should be able to decrease the likelihood of sending IPIs pretty much > > to zero. > > > > Or am I missing something here? > > I would recommend considering the following alternative for this tracing-rcu > flavor: > > - For all CPUs which are not nohz_full: > - Implement fast RCU read-side which only requires compiler barriers, > - Use IPIs to each of those CPUs when doing a grace period. > > - For all nohz_full CPUS: > - Dynamically detect CPUs which are nohz_full, > - Implement slower RCU read-side with memory barriers, > - No need to issue any IPI to those CPUs when doing the grace period. > > This should cover all use-cases: staying fast for the common case, without > disturbing RT workloads. > > Thoughts ? I will certainly add this to my list of potential solutions, and thank you for pointing me at it! Thanx, Paul
