On Thu, Feb 03, 2022 at 10:40:50AM +0900, Damien Le Moal wrote:
> On 2/1/22 06:13, Niklas Cassel wrote:
> > From: Niklas Cassel <niklas.cassel@xxxxxxx>
> >
> > Convert the stat code to report clat stats on a per priority granularity,
> > rather than simply supporting high/low priority.
> >
> > This is made possible by using the new clat_prio_stat array (per ddir),
> > together with the clat_prio_stat index which is saved in each io_u.
> >
> > The per priority samples are only printed when there are samples for more
> > than one priority in the clat_prio_stat array. If there are only samples
> > for one priority, that means that all I/Os where submitted using the same
> > priority, so no need to print.
> >
> > For example, running the following fio command:
> > fio --name=test --filename=/dev/sdc --direct=1 --runtime=60 --rw=randread \
> > --ioengine=io_uring --ioscheduler=mq-deadline --iodepth=32 --bs=32k \
> > --prioclass=2 --prio=7 --cmdprio_bssplit=32k/20/3/0:32k/10/1/4
> >
> > Now results in the following output:
> > test: (groupid=0, jobs=1): err= 0: pid=465655: Tue Feb 1 02:24:47 2022
> > read: IOPS=146, BW=4695KiB/s (4808kB/s)(276MiB/60239msec)
> > slat (usec): min=18, max=335, avg=62.87, stdev=22.59
> > clat (msec): min=2, max=2135, avg=217.97, stdev=287.26
> > lat (msec): min=2, max=2135, avg=218.03, stdev=287.26
> > clat prio 2/7 (msec): min=3, max=606, avg=106.57, stdev=86.64
> > clat prio 3/0 (msec): min=10, max=2135, avg=664.94, stdev=339.42
> > clat prio 1/4 (msec): min=2, max=300, avg=52.29, stdev=42.52
> > clat percentiles (msec):
> > | 1.00th=[ 8], 5.00th=[ 14], 10.00th=[ 19], 20.00th=[ 33],
> > | 30.00th=[ 52], 40.00th=[ 77], 50.00th=[ 108], 60.00th=[ 144],
> > | 70.00th=[ 192], 80.00th=[ 300], 90.00th=[ 684], 95.00th=[ 911],
> > | 99.00th=[ 1234], 99.50th=[ 1318], 99.90th=[ 1687], 99.95th=[ 1770],
> > | 99.99th=[ 2140]
> > clat prio 2/7 (69.25% of IOs) percentiles (msec):
> > | 1.00th=[ 7], 5.00th=[ 13], 10.00th=[ 17], 20.00th=[ 28],
> > | 30.00th=[ 44], 40.00th=[ 64], 50.00th=[ 85], 60.00th=[ 111],
> > | 70.00th=[ 140], 80.00th=[ 174], 90.00th=[ 226], 95.00th=[ 279],
> > | 99.00th=[ 368], 99.50th=[ 418], 99.90th=[ 502], 99.95th=[ 567],
> > | 99.99th=[ 609]
> > clat prio 3/0 (20.91% of IOs) percentiles (msec):
> > | 1.00th=[ 44], 5.00th=[ 138], 10.00th=[ 205], 20.00th=[ 347],
> > | 30.00th=[ 464], 40.00th=[ 558], 50.00th=[ 659], 60.00th=[ 760],
> > | 70.00th=[ 860], 80.00th=[ 961], 90.00th=[ 1099], 95.00th=[ 1217],
> > | 99.00th=[ 1485], 99.50th=[ 1687], 99.90th=[ 1871], 99.95th=[ 2140],
> > | 99.99th=[ 2140]
> > clat prio 1/4 (9.84% of IOs) percentiles (msec):
> > | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 13], 20.00th=[ 18],
> > | 30.00th=[ 24], 40.00th=[ 30], 50.00th=[ 39], 60.00th=[ 51],
> > | 70.00th=[ 63], 80.00th=[ 84], 90.00th=[ 114], 95.00th=[ 136],
> > | 99.00th=[ 188], 99.50th=[ 197], 99.90th=[ 300], 99.95th=[ 300],
> > | 99.99th=[ 300]
> > bw ( KiB/s): min= 3456, max= 5888, per=100.00%, avg=4697.60, stdev=472.38, samples=120
> > iops : min= 108, max= 184, avg=146.80, stdev=14.76, samples=120
> > lat (msec) : 4=0.11%, 10=2.57%, 20=8.67%, 50=18.21%, 100=18.34%
> > lat (msec) : 250=28.87%, 500=9.41%, 750=5.22%, 1000=5.09%, 2000=3.50%
> > lat (msec) : >=2000=0.01%
> > cpu : usr=0.16%, sys=0.97%, ctx=17715, majf=0, minf=262
> > IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=99.6%, >=64=0.0%
> > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
> > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
> > issued rwts: total=8839,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> > latency : target=0, window=0, percentile=100.00%, depth=32
>
> Nice !
> > @@ -2517,6 +2694,13 @@ void __show_run_stats(void)
> >
> > log_info_flush();
> > free(runstats);
> > +
> > + /* free arrays allocated by sum_thread_stats(), if any */
> > + for (i = 0; i < nr_ts; i++) {
> > + ts = &threadstats[i];
> > + if (!ts->disable_prio_stat)
> > + free_clat_prio_stats(ts);
>
> If disable_prio_stat is true, there will be no array to free so
> free_clat_prio_stats() will do nothing, no ? You could call
> free_clat_prio_stats() unconditionally to simplify.
Yes, free_clat_prio_stats() handles NULL, just like regular free().
I added the extra verbosity on purpose, to highlight the fact that this
allocation wasn't unconditionally performed for each struct thread_stat.
However, on second thought, it probably confuses more than it clarifies,
so I will remove it in V2.
Kind regards,
Niklas
