Returning to an old discussion... On Mon, Dec 12, 2022 at 01:29:24PM +0100, Oliver Neukum wrote: > > > On 08.12.22 18:40, Alan Stern wrote: > > On Thu, Dec 08, 2022 at 03:36:45PM +0100, Oliver Neukum wrote: > > > On 06.12.22 16:38, Alan Stern wrote: > > > It's hard to tell what's really going on. Looking at > > xpad_stop_output(), you see that it doesn't do anything if xpad->type is > > XTYPE_UNKNOWN. Is that what happened here? > > The output anchor in xpad was used. So I have to answer that in the negative. > > I can't figure out where the underlying race is. Maybe it's not > > directly connected with anchors after all. > > > > > As far as I can tell the order we decrease use_count is correct. But: > > > > > > 6ec4147e7bdbd (Hans de Goede 2013-10-09 17:01:41 +0200 1674) usb_anchor_resume_wakeups(anchor); > > > 94dfd7edfd5c9 (Ming Lei 2013-07-03 22:53:07 +0800 1675) atomic_dec(&urb->use_count); > > > > > > Do we need to guarantee memory ordering here? > > > > I don't think we need to do anything more. usb_kill_urb() is careful to > > wait for completion handlers to finish, and we already have > > By checking use_count > > > smp_mb__after_atomic() barriers in the appropriate places to ensure > > proper memory ordering. > > Do we? Looking at __usb_hcd_giveback_urb(): > > usb_unanchor_urb(urb); > > This is an implicit memory barrier > > if (likely(status == 0)) > usb_led_activity(USB_LED_EVENT_HOST); > > /* pass ownership to the completion handler */ > urb->status = status; > /* > * This function can be called in task context inside another remote > * coverage collection section, but kcov doesn't support that kind of > * recursion yet. Only collect coverage in softirq context for now. > */ > kcov_remote_start_usb_softirq((u64)urb->dev->bus->busnum); > urb->complete(urb); > kcov_remote_stop_softirq(); > > usb_anchor_resume_wakeups(anchor); > atomic_dec(&urb->use_count); > /* > * Order the write of urb->use_count above before the read > * of urb->reject below. Pairs with the memory barriers in > * usb_kill_urb() and usb_poison_urb(). > */ > smp_mb__after_atomic(); > > That is the latest time use_count can go to zero. > But what is the earliest time the CPU could reorder setting use_count to zero? > Try as I might the last certain memory barrier I can find in this function > is usb_unanchor_urb(). > That means another CPU can complete usb_kill_urb() before usb_anchor_resume_wakeups() > runs. > > usb_anchor_resume_wakeups(anchor); > > I think we need a memory barrier here, too. > > atomic_dec(&urb->use_count); Please comment on the proposed patch below. Alan Stern Index: usb-devel/drivers/usb/core/hcd.c =================================================================== --- usb-devel.orig/drivers/usb/core/hcd.c +++ usb-devel/drivers/usb/core/hcd.c @@ -1563,13 +1563,19 @@ int usb_hcd_submit_urb (struct urb *urb, usbmon_urb_submit_error(&hcd->self, urb, status); urb->hcpriv = NULL; INIT_LIST_HEAD(&urb->urb_list); - atomic_dec(&urb->use_count); /* - * Order the write of urb->use_count above before the read - * of urb->reject below. Pairs with the memory barriers in - * usb_kill_urb() and usb_poison_urb(). + * urb->use_count acts like a refcount, so decrementing it to + * 0 must be ordered after earlier accesses (pairs with the + * implicit control dependencies in the wait conditions of + * usb_kill_urb() and usb_poison_urb()). Also, the decrement + * must be ordered before the read of urb->reject below + * (pairs with the memory barriers in those same routines). + * + * Get the effect of full memory barriers before and after + * the decrement by using atomic_dec_return() instead of a + * simple atomic_dec(). */ - smp_mb__after_atomic(); + atomic_dec_return(&urb->use_count); atomic_dec(&urb->dev->urbnum); if (atomic_read(&urb->reject)) @@ -1672,13 +1678,19 @@ static void __usb_hcd_giveback_urb(struc kcov_remote_stop_softirq(); usb_anchor_resume_wakeups(anchor); - atomic_dec(&urb->use_count); /* - * Order the write of urb->use_count above before the read - * of urb->reject below. Pairs with the memory barriers in - * usb_kill_urb() and usb_poison_urb(). + * urb->use_count acts like a refcount, so decrementing it to + * 0 must be ordered after earlier accesses (pairs with the + * implicit control dependencies in the wait conditions of + * usb_kill_urb() and usb_poison_urb()). Also, the decrement + * must be ordered before the read of urb->reject below + * (pairs with the memory barriers in those same routines). + * + * Get the effect of full memory barriers before and after + * the decrement by using atomic_dec_return() instead of a + * simple atomic_dec(). */ - smp_mb__after_atomic(); + atomic_dec_return(&urb->use_count); if (unlikely(atomic_read(&urb->reject))) wake_up(&usb_kill_urb_queue); Index: usb-devel/drivers/usb/core/urb.c =================================================================== --- usb-devel.orig/drivers/usb/core/urb.c +++ usb-devel/drivers/usb/core/urb.c @@ -726,6 +726,10 @@ void usb_kill_urb(struct urb *urb) usb_hcd_unlink_urb(urb, -ENOENT); wait_event(usb_kill_urb_queue, atomic_read(&urb->use_count) == 0); + /* + * The test of urb->use_count creates a control dependency + * ordering the wait_event() call against any later writes. + */ atomic_dec(&urb->reject); } @@ -776,6 +780,10 @@ void usb_poison_urb(struct urb *urb) usb_hcd_unlink_urb(urb, -ENOENT); wait_event(usb_kill_urb_queue, atomic_read(&urb->use_count) == 0); + /* + * The test of urb->use_count creates a control dependency + * ordering the wait_event() call against any later writes. + */ } EXPORT_SYMBOL_GPL(usb_poison_urb);
