"Theodore Ts'o" <tytso@xxxxxxx> writes: > On Tue, Jun 15, 2021 at 08:17:55PM -0400, Gabriel Krisman Bertazi wrote: > Apologies for the delay in responding; somehow I missed your initial > e-mail on the subject on June 2nd, although I haven't found it in the > mailing list archives[1]. I don't know if it got caught in a spam > trap, or was accidentally deleted from my inbox. > > [1] https://lore.kernel.org/linux-ext4/87lf7rkffv.fsf@xxxxxxxxxxxxx/ > > I didn't do any bs=4k benchmarks before we landed the DIO iomap > changes, and it's interesting that it largely goes away with a 16k > block size[2] > > [2] https://people.collabora.com/~krisman/dio/week21/bench.png > > Looking at your flame graphs[3][4] > > [3] https://people.collabora.com/~krisman/dio/week23/clean_flames/5.4.0-dio_original-dio-ext4-write-4k.svg > [4] https://people.collabora.com/~krisman/dio/week23/clean_flames/5.5.0-dio_old-iomap-ext4-write-4k.svg > > ... nothing immediately jumps out at me. > > Have you compared the output of /proc/lock_stat for the two kernels? > And is there anything obvious in the blktrace of the two kernels? Hi Ted, I updated and rerun my test script [1] to collect lock_stat and blktrace. I particularly don't see anything standing out on lock-stat, but I'm sharing the raw data [2], in case you have a better insight. blktrace, on the other hand, shows something interesting. Looks like the original DIO is always keeping a lower device queue depth than iomap [3], although I'd expect the opposite if there is FS contention, correct? Iff ext4 is taking longer to submit bios, it would be expected for the device queue depth to be more empty instead of more full, right? [1] https://people.collabora.com/~krisman/dio/week24/bench-regression.sh [2] https://people.collabora.com/~krisman/dio/week24/ [3] (LARGE SVG!) https://people.collabora.com/~krisman/dio/week24/iowatcher-charts.svg -- Gabriel Krisman Bertazi
