> On Mon, Jun 26, 2017 at 04:19:27PM -0400, Don Zickus wrote: > > On Fri, Jun 23, 2017 at 11:50:25PM +0200, Thomas Gleixner wrote: > > > On Fri, 23 Jun 2017, Don Zickus wrote: > > > > Hmm, all this work for a temp fix. Kan, how much longer until the > > > > real fix of having perf count the right cycles? > > > > > > Quite a while. The approach is wilfully breaking the user space ABI, > > > which is not going to happen. > > > > > > And there is a simpler solution as well, as I said here: > > > > > > > > > http://lkml.kernel.org/r/alpine.DEB.2.20.1706221730520.1885@nanos > > > > Hi Thomas, > > > > So, you are saying instead of slowing down the perf counter, speed up > > the hrtimer to sample more frequently like so: > > > > diff --git a/kernel/watchdog.c b/kernel/watchdog.c index > > 03e0b69..8ff49de 100644 > > --- a/kernel/watchdog.c > > +++ b/kernel/watchdog.c > > @@ -160,7 +160,7 @@ static void set_sample_period(void) > > * and hard thresholds) to increment before the > > * hardlockup detector generates a warning > > */ > > - sample_period = get_softlockup_thresh() * ((u64)NSEC_PER_SEC / 5); > > + sample_period = get_softlockup_thresh() * ((u64)NSEC_PER_SEC / > 10); > > } > > Hi Kan, > > Will the above patch work for you? Hi Don & Thomas, Sorry for the late response. We just finished the tests for all proposed patches. There are three proposed patches so far. Patch 1: The patch as above which speed up the hrtimer. Patch 2: Thomas's first proposal. https://patchwork.kernel.org/patch/9803033/ https://patchwork.kernel.org/patch/9805903/ Patch 3: my original proposal which increase the NMI watchdog timeout by 3X https://patchwork.kernel.org/patch/9802053/ According to our test, only patch 3 works well. The other two patches will hang the system eventually. For patch 1, the system hang after running our test case for ~1 hour. For patch 2, the system hang in running the overnight test. There is no error message shown when the system hang. So I don't know the root cause yet. BTW: We set 1 to watchdog_thresh when we did the test. It's believed that can speed up the failure. Thanks, Kan