On Thu, 2022-07-21 at 16:06 +0200, Lukas Czerner wrote: > On Wed, Jul 20, 2022 at 12:42:11PM -0400, Jeff Layton wrote: > > On Wed, 2022-07-20 at 17:22 +0200, Lukas Czerner wrote: > > > > > But not zero, at least > > > every time the inode is loaded from disk it is scheduled for i_version > > > update on the next attempted increment. Could that have an effect on > > > some particular common workload you can think of? > > > > > > > FWIW, it's doubtful that you'd even notice this. You'd almost certainly > > be updating the mtime or ctime on the next change anyway, so updating > > the i_version in that case is basically free. You will probably need to > > do some a few extra atomic in-memory operations, but that's probably not > > noticeable in something I/O constrained. > > > > > > > > Could you provide some performance numbers for iversion case? > > > > > > > I'm writing to a LVM volume on a no-name-brand ssd I have sitting > > around. fio jobfile is here: > > That's very simplistic test, but fair enough. I've ran 10 iterations of > xfstests with and without iversion and there is no significant > difference, in fact it's all well within run by run variation. That's > true in aggregate as well for individual tests. > Yeah. This change was most evident with small I/O sizes, so if there is an effect here it'll likely show up there. > However there are problems to solve before we attempt to make it a > default. With -o iversion ext4/026 and generic/622 fails. The ext4/026 > seems to be a real bug and I am not sure about the other one yet. > > I'll look into it. > Interesting, thanks. Lack of testing with that option enabled is probably another good reason to go ahead and make it the default. Let me know what you find. > -Lukas > > > > > [global] > > name=fio-seq-write > > filename=fio-seq-write > > rw=write > > bs=4k > > direct=0 > > numjobs=1 > > time_based > > runtime=300 > > > > [file1] > > size=1G > > ioengine=libaio > > iodepth=16 > > > > iversion support disabled: > > > > $ fio ./4k-write.fio > > file1: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 > > fio-3.27 > > Starting 1 process > > file1: Laying out IO file (1 file / 1024MiB) > > Jobs: 1 (f=1): [W(1)][100.0%][w=52.5MiB/s][w=13.4k IOPS][eta 00m:00s] > > file1: (groupid=0, jobs=1): err= 0: pid=10056: Wed Jul 20 12:28:21 2022 > > write: IOPS=96.3k, BW=376MiB/s (394MB/s)(110GiB/300001msec); 0 zone resets > > slat (nsec): min=1112, max=5727.5k, avg=1917.70, stdev=1300.30 > > clat (nsec): min=1112, max=2146.5M, avg=156067.38, stdev=15568002.13 > > lat (usec): min=3, max=2146.5k, avg=158.03, stdev=15568.00 > > clat percentiles (usec): > > | 1.00th=[ 36], 5.00th=[ 36], 10.00th=[ 37], 20.00th=[ 37], > > | 30.00th=[ 38], 40.00th=[ 38], 50.00th=[ 38], 60.00th=[ 39], > > | 70.00th=[ 39], 80.00th=[ 40], 90.00th=[ 42], 95.00th=[ 44], > > | 99.00th=[ 52], 99.50th=[ 59], 99.90th=[ 77], 99.95th=[ 88], > > | 99.99th=[ 169] > > bw ( KiB/s): min=15664, max=1599456, per=100.00%, avg=897761.07, stdev=504329.17, samples=257 > > iops : min= 3916, max=399864, avg=224440.26, stdev=126082.33, samples=257 > > lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=98.80% > > lat (usec) : 100=1.18%, 250=0.02%, 500=0.01% > > lat (msec) : 10=0.01%, 2000=0.01%, >=2000=0.01% > > cpu : usr=5.45%, sys=23.92%, ctx=78418, majf=0, minf=14 > > IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% > > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% > > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% > > issued rwts: total=0,28889786,0,0 short=0,0,0,0 dropped=0,0,0,0 > > latency : target=0, window=0, percentile=100.00%, depth=16 > > > > Run status group 0 (all jobs): > > WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), io=110GiB (118GB), run=300001-300001msec > > > > Disk stats (read/write): > > dm-7: ios=0/22878, merge=0/0, ticks=0/373254, in_queue=373254, util=43.89%, aggrios=0/99746, aggrmerge=0/9246, aggrticks=0/1406831, aggrin_queue=1408420, aggrutil=73.56% > > sda: ios=0/99746, merge=0/9246, ticks=0/1406831, in_queue=1408420, util=73.56% > > > > mounted with -o iversion: > > > > $ fio ./4k-write.fio > > file1: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 > > fio-3.27 > > Starting 1 process > > Jobs: 1 (f=1): [W(1)][100.0%][eta 00m:00s] > > file1: (groupid=0, jobs=1): err= 0: pid=10369: Wed Jul 20 12:33:57 2022 > > write: IOPS=96.2k, BW=376MiB/s (394MB/s)(110GiB/300001msec); 0 zone resets > > slat (nsec): min=1112, max=1861.5k, avg=1994.58, stdev=890.78 > > clat (nsec): min=1392, max=2113.3M, avg=156252.71, stdev=15409487.99 > > lat (usec): min=3, max=2113.3k, avg=158.30, stdev=15409.49 > > clat percentiles (usec): > > | 1.00th=[ 37], 5.00th=[ 38], 10.00th=[ 38], 20.00th=[ 38], > > | 30.00th=[ 39], 40.00th=[ 39], 50.00th=[ 40], 60.00th=[ 40], > > | 70.00th=[ 41], 80.00th=[ 42], 90.00th=[ 43], 95.00th=[ 45], > > | 99.00th=[ 53], 99.50th=[ 60], 99.90th=[ 79], 99.95th=[ 90], > > | 99.99th=[ 174] > > bw ( KiB/s): min= 304, max=1540000, per=100.00%, avg=870727.42, stdev=499371.78, samples=265 > > iops : min= 76, max=385000, avg=217681.82, stdev=124842.94, samples=265 > > lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=98.49% > > lat (usec) : 100=1.48%, 250=0.02%, 500=0.01% > > lat (msec) : 2=0.01%, 2000=0.01%, >=2000=0.01% > > cpu : usr=5.71%, sys=24.49%, ctx=52874, majf=0, minf=18 > > IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0% > > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% > > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% > > issued rwts: total=0,28856695,0,0 short=0,0,0,0 dropped=0,0,0,0 > > latency : target=0, window=0, percentile=100.00%, depth=16 > > > > Run status group 0 (all jobs): > > WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), io=110GiB (118GB), run=300001-300001msec > > > > Disk stats (read/write): > > dm-7: ios=1/16758, merge=0/0, ticks=2/341817, in_queue=341819, util=47.93%, aggrios=1/98153, aggrmerge=0/5691, aggrticks=2/1399496, aggrin_queue=1400893, aggrutil=73.42% > > sda: ios=1/98153, merge=0/5691, ticks=2/1399496, in_queue=1400893, util=73.42% > > > > -- > > Jeff Layton <jlayton@xxxxxxxxxx> > > > -- Jeff Layton <jlayton@xxxxxxxxxx>
