>>>>>> Looks like the hlist change is probably the issue, though it specifically >>>>>> uses: >>>>>> >>>>>> #define hlist_entry_safe(ptr, type, member) \ >>>>>> (ptr) ? hlist_entry(ptr, type, member) : NULL >>>>>> >>>>>> I'm still looking at the code in question and it's assembly, but I can't >>>>>> figure out what's going wrong. I was also trying to see what's so special >>>>>> about this loop in find_pid_ns as opposed to the rest of the kernel code >>>>>> that uses hlist_for_each_entry_rcu() but couldn't find out why. >>>>>> >>>>>> Is it somehow possible that if we rcu_dereference_raw() the same thing twice >>>>>> inside the same rcu_read_lock() section we'll get different results? That's >>>>>> really the only reason for this crash that comes to mind at the moment, very >>>>>> unlikely - but that's all I have right now. >>>>>> >>>>> >>>>> Yep >>>>> >>>>> #define hlist_entry_safe(ptr, type, member) \ >>>>> (ptr) ? hlist_entry(ptr, type, member) : NULL >>>>> >>>>> Is not safe, as ptr can be evaluated twice, and thats not good at all... >>>> >>>> ptr is being evaluated twice, but in this case this is an >>>> rcu_dereference_raw() value within the same rcu_read_lock() section. >>>> >>>> Is it still problematic? >>> >>> Definitely. >>> >>> Head in this instance the expression: &pid_hash[pid_hashfn(nr, ns)] >>> >>> And the crash clearly shows that when hilst_entry is being evaluated the >>> HEAD is NULL. >> >> Okay, I'm even more confused now. >> >> The expression in question is: >> >> hlist_entry_safe(rcu_dereference_bh(hlist_first_rcu(head))) >> >> You're saying that "rcu_dereference_bh(hlist_first_rcu(head))" can change between >> the two evaluations we do. That would mean that 'head' has changed in between, right? >> >> In that case, the list itself has changed - which means that RCU has changed the >> list underneath us. >> >> hlist_first_rcu() doesn't have any side-effects, it doesn't modify the list whatsoever, >> so the only thing that can change is 'head'. Why is it allowed to change if the list >> is protected by RCU? > > RCU does not prevent the list from changing. Instead, it prevents anything > that was in the list from being freed during a given RCU read-side critical > section. Here is how it is supposed to happen: > > head---->A > > Task 1 picks up the pointer from head to A, and sees that it is non-NULL. > > Task 2 removes A from the list, so that the pointer from head is now NULL: > > head A > | > | > V > NULL > > Now task 1 refetches from head, and is fatally disappointed to get a > NULL pointer. > > Now, had task 1 avoided the refetch, it would be still working with > a pointer to A. Since A won't be freed until the end of an RCU grace > period, all would have been well. Again, one way to handle this is > as follows: > > #define hlist_entry_safe(ptr, type, member) \ > ({ typeof(ptr) ____ptr = (ptr); \ > ____ptr ? hlist_entry(____ptr, type, member) : NULL; \ > }) > > This way, "ptr" is executed exactly once, and the check and the > hlist_entry() are both using the same value. > I just played with trinity, and triggered this bug in just a few mins, and I tried Paul's proposed fix and it works. _______________________________________________ Containers mailing list Containers@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/containers