On Thu, Jan 23, 2020 at 12:59 PM Evan Green <evgreen@xxxxxxxxxxxx> wrote: > > On Thu, Jan 23, 2020 at 10:17 AM Thomas Gleixner <tglx@xxxxxxxxxxxxx> wrote: > > > > Evan, > > > > Thomas Gleixner <tglx@xxxxxxxxxxxxx> writes: > > > This is not yet debugged fully and as this is happening on MSI-X I'm not > > > really convinced yet that your 'torn write' theory holds. > > > > can you please apply the debug patch below and run your test. When the > > failure happens, stop the tracer and collect the trace. > > > > Another question. Did you ever try to change the affinity of that > > interrupt without hotplug rapidly while the device makes traffic? If > > not, it would be interesting whether this leads to a failure as well. > > Thanks for the patch. Looks pretty familiar :) > I ran into issues where trace_printks on offlined cores seem to > disappear. I even made sure the cores were back online when I > collected the trace. So your logs might not be useful. Known issue > with the tracer? > > I figured I'd share my own debug chicken scratch, in case you could > glean anything from it. The LOG entries print out timestamps (divide > by 1000000) that you can match up back to earlier in the log (ie so > the last XHCI MSI change occurred at 74.032501, the last interrupt > came in at 74.032405). Forgive the mess. > > I also tried changing the affinity rapidly without CPU hotplug, but > didn't see the issue, at least not in the few minutes I waited > (normally repros easily within 1 minute). An interesting datapoint. One additional datapoint. The intel guys suggested enabling CONFIG_IRQ_REMAP, which does seem to eliminate the issue for me. I'm still hoping there's a smaller fix so I don't have to add all that in. -Evan
