On Mon, Feb 13, 2023 at 01:46:11PM -0500, Kent Overstreet wrote:
> On Mon, Feb 13, 2023 at 10:24:13AM +0100, Peter Zijlstra wrote:
> > On Sun, Feb 12, 2023 at 10:23:44AM -0500, Alan Stern wrote:
> > > Provided it acquires the parent device's lock first, this is
> > > utterly safe no matter what order the children are locked in. Try
> > > telling that to lockdep!
> >
> > mutex_lock_next_lock(child->lock, parent->lock) is there to express this
> > exact pattern, it allows taking multiple child->lock class locks (in any
> > order) provided parent->lock is held.
>
> Perhaps I'm stupid, but I've never understood how subclasses - or this -
> are supposed to work.
>
> Locks don't get a fixed subclass, so what's to prevent some code from
> going
So there's two annotations here, the nest_lock thing and subclasses,
they're distinct things.
Every class gets a fixed 8 subclasses (0-7) given by the unique byte
addresses inside the actual key object.
Subclasses will let you create nesting order of the same class that are
acceptable. Typically lock/1 nests inside lock/0, but that's not
hard-coded, simply convention.
The way it is used is given an external lock order, say the CPU number
for the runqueue locks, you do things like:
void double_rq_lock(struct rq *rq1, struct rq *r2)
{
lockdep_assert_irqs_disabled();
if (rq_order_less(r2, rq1))
swap(rq1, rq2);
raw_spin_rq_lock(rq1);
if (__rq_lockp(rq1) != __rq_lock(rq2))
raw_spin_rq_lock_nested(rq2, SINGLE_DEPTH_NESTING);
...
}
(which is more complicated than it needs to be due to the whole
core-scheduling mess, but should still be readable I suppose).
Basically we make sure rq1 and rq2 are in the correct order and acquire
them with subclass 0 (the default) and subcless 1 (SINGLE_DEPTH_NESTING)
resp. dictating the subclass order.
This is lock order per decree, if you get the order function wrong
lockdep will not see the inversion but you *will* deadlock.
Then there's that nesting lock, that requires two classes and at least 3
locks to make sense:
P, C1, C2
Where we posit that any multi-lock of Cn is fully serialized by P and it
is used like:
mutex_lock(P);
mutex_lock_nest_lock(C1, P);
mutex_lock_nest_lock(C2, P);
Where any order of Cn is acceptable, because fully ordered by P.
If you were to combine this with subclass on Cn to allow multi-lock
instances not order by P, you get to keep the pieces :-)
