On Fri, Jul 19, 2019 at 04:50:11PM +0900, Byungchul Park wrote: > On Thu, Jul 18, 2019 at 05:09:45PM -0400, Joel Fernandes wrote: > > Hello friends, > > > > Just providing an update on my debugging of percpu_rwsem (related to > > rcu-sync) for the day! which I pinged Byungchul about. Please ignore > > this email if you are busy :) I am just archiving it in here.. > > > > As you may know, percpu_rwsem uses rcu-sync framework to reduce cost > > of read-side by making it free of any serializing/atomic instructions > > at all. However, there was one sempahore which broke the rules! > > > > I spent a couple hours trying to figure out why > > cgroup_threadgroup_rwsem always entered the reader-slow path on my > > system (RCU-sync turns out to be non-idle for this rwsem). I really > > thought it was a bug, because I felt what's the pointed of rcu-sync if > > it never goes idle.. > > Yes, with the following patch, the cgroup rwsem cannot make use of > rcu_sync any more, but it still gets benefit from percpu structure > as you told me like avoiding cache bouncing and contention on a shared > area even though every read lock keeps firing smp full barrier. Yes. So it seems to me main benefit of RCU in percpu_rw_sempahore is to completely avoid memory barriers in the read path, while also benefiting from the percpu nature of the lock. > What matters is which one is more expensive between (1) firing smp_mb > and (2) accessing a shared data, sem->count, and acquiring/releasing > sem->wait_lock. I think using percpu-rwsem involving the smp barrier is > much better even with rcu_sync disabled. Right. Fully agreed. thanks, - Joel
