Hi, Thank you for fio. Apologies if this is the wrong place to ask. I am running fio on a laptop with an nvme ssd and have a question about differences between results. test.1 - test.8 are each 900 Mb files of random data created beforehand run 1 - echo 3 | sudo tee /proc/sys/vm/drop_caches fio --name=fiotest --filename=test.1:test.2:test.3:test.4:test.5:test.6:test.7:test.8 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=8 --invalidate=0 --ioengine=psync results: iops : min= 6794, max=86407, avg=26989.80, stdev=33609.35, samples=5 iops : min= 6786, max=84998, avg=26818.40, stdev=32972.35, samples=5 iops : min= 6816, max=83273, avg=26422.20, stdev=32200.02, samples=5 iops : min= 6838, max=88227, avg=27462.20, stdev=34405.56, samples=5 iops : min= 6868, max=85788, avg=26870.00, stdev=33332.10, samples=5 iops : min= 6818, max=84768, avg=26719.60, stdev=32880.23, samples=5 iops : min= 6810, max=86326, avg=27005.60, stdev=33581.89, samples=5 iops : min= 6856, max=81411, avg=26165.00, stdev=31345.85, samples=5 READ: bw=2644MiB/s (2773MB/s), 331MiB/s-332MiB/s (347MB/s-348MB/s), io=7200MiB (7550MB), run=2709-2723msec run 2 - echo 3 | sudo tee /proc/sys/vm/drop_caches fio --name=fiotest1 --filename=test.1 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest2 --filename=test.2 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest3 --filename=test.3 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest4 --filename=test.4 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest5 --filename=test.5 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest6 --filename=test.6 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest7 --filename=test.7 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & fio --name=fiotest8 --filename=test.8 -gtod_reduce=1 --size=900M --rw=randread --bs=8K --direct=0 --numjobs=1 --invalidate=0 --ioengine=psync & wait results (from each) - iops : min= 5836, max= 6790, avg=6481.37, stdev=140.90, samples=35 READ: bw=50.6MiB/s (53.1MB/s), 50.6MiB/s-50.6MiB/s (53.1MB/s-53.1MB/s), io=900MiB (944MB), run=17777-17777msec iops : min= 5924, max= 6784, avg=6476.54, stdev=126.85, samples=35 READ: bw=50.6MiB/s (53.1MB/s), 50.6MiB/s-50.6MiB/s (53.1MB/s-53.1MB/s), io=900MiB (944MB), run=17785-17785msec iops : min= 5878, max= 6766, avg=6475.23, stdev=138.57, samples=35 READ: bw=50.6MiB/s (53.0MB/s), 50.6MiB/s-50.6MiB/s (53.0MB/s-53.0MB/s), io=900MiB (944MB), run=17792-17792msec iops : min= 5866, max= 6788, avg=6478.74, stdev=139.18, samples=35 READ: bw=50.6MiB/s (53.1MB/s), 50.6MiB/s-50.6MiB/s (53.1MB/s-53.1MB/s), io=900MiB (944MB), run=17782-17782msec iops : min= 5890, max= 6782, avg=6477.91, stdev=135.21, samples=35 READ: bw=50.6MiB/s (53.1MB/s), 50.6MiB/s-50.6MiB/s (53.1MB/s-53.1MB/s), io=900MiB (944MB), run=17783-17783msec iops : min= 5838, max= 6790, avg=6450.11, stdev=137.17, samples=35 READ: bw=50.3MiB/s (52.7MB/s), 50.3MiB/s-50.3MiB/s (52.7MB/s-52.7MB/s), io=900MiB (944MB), run=17892-17892msec iops : min= 5922, max= 6750, avg=6454.06, stdev=125.76, samples=35 READ: bw=50.3MiB/s (52.8MB/s), 50.3MiB/s-50.3MiB/s (52.8MB/s-52.8MB/s), io=900MiB (944MB), run=17880-17880msec iops : min= 5916, max= 6724, avg=6465.94, stdev=127.54, samples=35 READ: bw=50.5MiB/s (52.0MB/s), 50.5MiB/s-50.5MiB/s (52.0MB/s-52.0MB/s), io=900MiB (944MB), run=17816-17816msec Question: if I am really doing 900 MiB of random reads from 8 separate files in both tests, and the page-cache was empty before each test, does this difference in performance make sense or am I reading the results incorrectly ? thank you for any info/ advice, suggestions, -mark
