Hi All, In commit "Add support for doing total latency percentiles" (b599759ba565e7f2f573af364e6da4fe6d556a90), the total latency percentiles option was added. However, some workloads appear broken: # fio --name=test --ioengine=libaio --direct=1 --rw=randread --iodepth=1 --size=100% --bs=4k --filename=/dev/nvme0n1 --runtime=5s --write_lat_log=test --disable_lat=0 --disable_clat=1 --disable_slat=1 --lat_percentiles=1 test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 fio-3.2-66-g67bfe Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=351MiB/s,w=0KiB/s][r=89.9k,w=0 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=6879: Mon Dec 4 15:46:04 2017 read: IOPS=89.7k, BW=351MiB/s (368MB/s)(1753MiB/5001msec) lat (usec): min=9, max=2259, avg=10.67, stdev= 7.66 lat percentiles (nsec): | 1.00th=[ 0], 5.00th=[ 0], 10.00th=[ 0], 20.00th=[ 0], | 30.00th=[ 0], 40.00th=[ 0], 50.00th=[ 0], 60.00th=[ 0], | 70.00th=[ 0], 80.00th=[ 0], 90.00th=[ 0], 95.00th=[ 0], | 99.00th=[ 0], 99.50th=[ 0], 99.90th=[ 0], 99.95th=[ 0], | 99.99th=[ 0] In the above, we try to enable only lat log, so disable clat and slat. The summary output shows lat, but the values are zero. # fio --name=test --ioengine=libaio --direct=1 --rw=randread --iodepth=1 --size=100% --bs=4k --filename=/dev/nvme0n1 --runtime=5s --write_lat_log=test --disable_lat=0 --disable_clat=0 --disable_slat=1 --lat_percentiles=1 test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 fio-3.2-66-g67bfe Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=356MiB/s,w=0KiB/s][r=91.1k,w=0 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=6956: Mon Dec 4 15:47:43 2017 read: IOPS=87.8k, BW=343MiB/s (360MB/s)(1715MiB/5001msec) clat (nsec): min=561, max=2258.3k, avg=9941.30, stdev=6660.37 lat (usec): min=9, max=2259, avg=10.88, stdev= 7.52 lat percentiles (nsec): | 1.00th=[10304], 5.00th=[10304], 10.00th=[10304], 20.00th=[10304], | 30.00th=[10432], 40.00th=[10432], 50.00th=[10432], 60.00th=[10432], | 70.00th=[10432], 80.00th=[10944], 90.00th=[11072], 95.00th=[14784], | 99.00th=[15296], 99.50th=[15424], 99.90th=[16064], 99.95th=[17024], | 99.99th=[24704] In the above, we force clat on, and now the lat values are non-zero in the summary output. In the commit for stat.c, I see: if (ts->lat_percentiles) add_clat_percentile_sample(ts, nsec, ddir); Does that seem correct? Regards, Jeff -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html