Hi,
Let me chime in this interesting thread.
On Thu, 9 Mar 2023 13:53:39 -0800, Paul E. McKenney wrote:
> On Thu, Mar 09, 2023 at 03:17:09PM +0000, Zhuo, Qiuxu wrote:
>> > From: Paul E. McKenney <paulmck@xxxxxxxxxx>
>> > [...]
>> > >
>> > > a's standard deviation is ~0.4.
>> > > b's standard deviation is ~0.5.
>> > >
>> > > a's average 9.0 is at the upbound of the standard deviation of b's [8.0, 9].
>> > > So, the measurements should be statistically significant to some degree.
>> >
>> > That single standard deviation means that you have 68% confidence that the
>> > difference is real. This is not far above the 50% leval of random noise.
>> > 95% is the lowest level that is normally considered to be statistically
>> > significant.
>>
>> 95% means there is no overlap between two standard deviations of a
>> and two standard deviations of b.
>>
>> This relies on either much less noise during testing or a big enough
>> difference between a and b.
Appended is a histogram comparing 2 data sets.
As you see, the one with v2 patch is far from normal distribution.
I think there is at least two peaks.
The one at the right around 9.7 seems not affected by the patch.
In such a case, average and standard deviation of all the data don't
tell much.
It is hard to say anything for sure with such small set of samples.
And the shape of the plot is likely to be highly dependent on machine
setups.
Hope this helps.
Thanks, Akira
>>
[...]Attachment:
time-to-boot.png
Description: PNG image
