Hello Ted,
On 1/15/20 10:18 PM, Theodore Y. Ts'o wrote:
On Wed, Jan 15, 2020 at 05:00:53AM +0530, Ritesh Harjani wrote:
I too collected some performance numbers on my x86 box with
--direct=1, bs=4K/1M & ioengine=libaio, with default opt v/s dioread_nolock
opt on latest ext4 git tree.
I found the delta to be within +/- 6% in all of the runs which includes, seq
read, mixed rw & mixed randrw.
Thanks for taking the performance measurements!
Are you able to release more detail of what the deltas were for those
tests? And how stable were those results across repeated runs?
I have collected these numbers using fio. All of the column values
shown in the attachment & pasted below are average of 3 runs.
I checked all those individual runs too and saw that even in the run-to-
run variations the delta was within +/- 5% only.
The naming of those individual runs *.json files are a bit weird and
will take sometime if I have to publish the individual runs. But do let
me know if that as well is required. I can make some changes in the
scripts to print individual run's numbers too.
I have pasted the individual fio files too which I used for my testing.
I have used libaio as ioengine (with iodepth=16) in the SeqRead case and
psync in others.
Performance numbers:
===================
(seq) read (4K) - libaio
============
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- --------------------- ----------------------------
1 138.928059895833 138.824869791667
8 129.345052083333 124.472005208333
24 71.6555989583333 72.275390625
(Not sure why 24 thread case has lower perf numbers, but on increasing
the iodepth=32, I could see an increase in bw with 24 threads case
too.)
(seq) read (1M) - libaio
===========
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- --------------------- ---------------
1 138.905598958333 138.832682291667
8 111.263997395833 109.301106770833
24 121.895182291667 127.75390625
randrw(4K) - read (psync)
==========================
threads default_opt_read [KB/s] dioread_opt_read [KB/s]
---------- ---------------- ----------------
1 414.666666666667 410.0
8 780.0 792.333333333333
24 967.333333333333 991.666666666667
randrw(4K) - write (psync)
===========================
threads default_opt_write [KB/s] dioread_opt_write [KB/s]
---------- ----------------- -----------------
1 418.0 413.666666666667
8 796.666666666667 809.666666666667
24 981.333333333333 1007.66666666667
randrw(1M) - read (psync)
=================
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- ----------------- -----------------------
1 39.5693359375 39.7288411458333
8 44.5179036458333 47.9098307291667
24 50.2861328125 51.720703125
randrw(1M) - write (psync)
==================
threads default_opt_write(MB/s) dioread_nolock_opt_write(MB/s)
---------- ---------------------- -----------------------
1 41.4583333333333 41.5068359375
8 46.0768229166667 49.568359375
24 49.5947265625 50.7083333333333
rw(1M) - read (psync)
=============
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- ---------------------- -------------------
1 43.6458333333333 43.6770833333333
8 48.1178385416667 49.2718098958333
24 50.5703125 53.7890625
rw(1M) - write (psync)
==============
threads default_opt_write(MB/s) dioread_nolock_opt_write(MB/s)
---------- ----------------------- ------------------
1 45.5065104166667 45.5654296875
8 49.7431640625 51.0690104166667
24 50.2493489583333 53.3463541666667
FIO FILES
=========
dio_read.fio
============
[global]
ioengine=libaio
rw=read
runtime=60
iodepth=16
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
dio_randrw.fio
==============
[global]
ioengine=psync
rw=randrw
runtime=60
iodepth=1
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
dio_rw.fio
==========
[global]
ioengine=psync
rw=rw
runtime=60
iodepth=1
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
Thanks
-ritesh
SeqRead (4K) - libaio
============
threads default_opt[read_4K(MB/s)] dioread_nolock_opt [read_4K(MB/s)]
---------- --------------------------- ------------------------------
1 138.928059895833 138.824869791667
8 129.345052083333 124.472005208333
24 71.6555989583333 72.275390625
SeqRead(1M) - libaio
===========
threads default_opt [read_1M(MB/s)] dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- --------------------------------
1 138.905598958333 138.832682291667
8 111.263997395833 109.301106770833
24 121.895182291667 127.75390625
randrw(4K) - read (psync)
==========================
threads default_opt_read [KB/s] dioread_opt_read [KB/s]
---------- ---------------- ----------------
1 414.666666666667 410.0
8 780.0 792.333333333333
24 967.333333333333 991.666666666667
randrw(4K) - write (psync)
===========================
threads default_opt_write [KB/s] dioread_opt_write [KB/s]
---------- ----------------- -----------------
1 418.0 413.666666666667
8 796.666666666667 809.666666666667
24 981.333333333333 1007.66666666667
randrw(1M) - read (psync)
=================
threads default_opt [read_1M(MB/s) dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- ---------------------------------
1 39.5693359375 39.7288411458333
8 44.5179036458333 47.9098307291667
24 50.2861328125 51.720703125
randrw(1M) - write (psync)
==================
threads default_opt [write_1M(MB/s)] dioread_nolock_opt [write_1M(MB/s)]
---------- ---------------------------- ----------------------------------
1 41.4583333333333 41.5068359375
8 46.0768229166667 49.568359375
24 49.5947265625 50.7083333333333
rw(1M) - read (psync)
=============
threads default_opt [read_1M(MB/s)] dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- ---------------------------------
1 43.6458333333333 43.6770833333333
8 48.1178385416667 49.2718098958333
24 50.5703125 53.7890625
rw(1M) - write (psync)
==============
threads default_opt [write_1M(MB/s)] dioread_nolock_opt [write_1M(MB/s)
---------- ----------------------------- ---------------------------------
1 45.5065104166667 45.5654296875
8 49.7431640625 51.0690104166667
24 50.2493489583333 53.3463541666667
