Hi Alan,
On 7/23/21 3:05 PM, Alan Stern wrote:
On Fri, Jul 23, 2021 at 08:52:50AM +0200, Manfred Spraul wrote:
Hi Alan,
Hi.
On 7/23/21 4:08 AM, Alan Stern wrote:
On Wed, Jul 21, 2021 at 02:10:01PM -0700, Paul E. McKenney wrote:
This commit adds example code for heuristic lockless reads, based loosely
on the sem_lock() and sem_unlock() functions.
Reported-by: Manfred Spraul <manfred@xxxxxxxxxxxxxxxx>
[ paulmck: Update per Manfred Spraul and Hillf Danton feedback. ]
Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxx>
---
.../Documentation/access-marking.txt | 94 +++++++++++++++++++
1 file changed, 94 insertions(+)
diff --git a/tools/memory-model/Documentation/access-marking.txt b/tools/memory-model/Documentation/access-marking.txt
index 58bff26198767..be7d507997cf8 100644
--- a/tools/memory-model/Documentation/access-marking.txt
+++ b/tools/memory-model/Documentation/access-marking.txt
@@ -319,6 +319,100 @@ of the ASSERT_EXCLUSIVE_WRITER() is to allow KCSAN to check for a buggy
concurrent lockless write.
+Lock-Protected Writes With Heuristic Lockless Reads
+---------------------------------------------------
+
+For another example, suppose that the code can normally make use of
+a per-data-structure lock, but there are times when a global lock
+is required. These times are indicated via a global flag. The code
+might look as follows, and is based loosely on nf_conntrack_lock(),
+nf_conntrack_all_lock(), and nf_conntrack_all_unlock():
+
+ bool global_flag;
+ DEFINE_SPINLOCK(global_lock);
+ struct foo {
+ spinlock_t f_lock;
+ int f_data;
+ };
+
+ /* All foo structures are in the following array. */
+ int nfoo;
+ struct foo *foo_array;
+
+ void do_something_locked(struct foo *fp)
+ {
+ bool gf = true;
+
+ /* IMPORTANT: Heuristic plus spin_lock()! */
+ if (!data_race(global_flag)) {
+ spin_lock(&fp->f_lock);
+ if (!smp_load_acquire(&global_flag)) {
+ do_something(fp);
+ spin_unlock(&fp->f_lock);
+ return;
+ }
+ spin_unlock(&fp->f_lock);
+ }
+ spin_lock(&global_lock);
+ /* Lock held, thus global flag cannot change. */
+ if (!global_flag) {
How can global_flag ever be true at this point? The only line of code
that sets it is in begin_global() below, it only runs while global_lock
is held, and global_flag is set back to false before the lock is
released.
It can't be true. The code is a simplified version of the algorithm in
ipc/sem.c.
For the ipc/sem.c, global_flag can remain true even after dropping
global_lock.
When transferring the approach to nf_conntrack_core, I didn't notice that
nf_conntrack doesn't need a persistent global_flag.
Thus the recheck after spin_lock(&global_lock) is not needed.
In fact, since global_flag is true if and only if global_lock is locked,
perhaps it can be removed entirely and replaced with
spin_is_locked(&global_lock).
I try to avoid spin_is_locked():
- spin_is_locked() is no memory barrier
- spin_lock() is an acquire memory barrier - for the read part. There is
no barrier at all related to the write part.
With an explicit variable, the memory barriers can be controlled much
better - and it is guaranteed to work in the same way on all architectures.
--
Manfred
