On 2/9/23 02:08, Shin'ichiro Kawasaki wrote:
When zonemode=zbd is specified, fio does 'sectors with data' accounting to
record the total number of sectors that have been written on a zoned block
device. This accounting has issues as follows:
Issue 1)
The name 'sectors with data' indicates that the accounting uses 'sector' as
the unit. However, it is implemented using 'byte' as the unit.
Issue 2)
The accounting does not work correctly when multi-jobs with different IO
ranges due to two reasons. One reason is the accounting field initialization.
The accounting field is shared across all jobs, but it is initialized
repeatedly by all jobs. This results in wrong behaviors except the last job.
The second reason is definition difference between the zone_reset_threshold
option and the accounting. The option is defined as the ratio of valid
written data to the IO range of each job. However, the accounting is done
per device and shared across all jobs. This coverage gap between each job's
IO range and the accounting range causes unexpected zone reset.
Issue 3)
Counting the total number of written sectors requires taking the zone lock of
all zones in a job IO range. For a multi-job workload with overlapping IO
ranges, this often leads to significant zone lock contention, resulting in
some jobs starting IOs only after other jobs have completed their work
(which looks like an apparent deadlock on startup).
This series addresses the issues with solutions as follows:
For Issue 1)
Rephrase variables, comments and man pages to indicate that the accounting
unit is not sector. -> 3rd and 4th patches
For Issue 2)
Limit the condition of the accounting to ensure correct accounting. Do the
accounting only when the zone_reset_threshold option is specified and all
write jobs have the same IO range. Initialize the accounting field only once
for the 1st job. -> 5th and 6th patches
For Issue 3)
Move the total valid bytes counting code from "file reset" after job start
to "file set up" before job start. This allows to count without zone locks,
then avoids the lock contention. -> 7th patch
The first two patches are preparation patches to reduce references to the
'sectors with data' accounting field. The last 8th patch adds test cases for the
zone_reset_threshold option.
Changes from v2:
* 4th patch: rephrased to cover the case that IO range has conventional zones
* 6th patch: added a check to ensure f->min_zone and f->max_zones are different
* 7th patch: renamed zbd_set_vdb() to zbd_verify_and_set_vdb()
* Reflected other comments on the list and added Reviewed-by tags
Changes from v1:
* Reworked not to change the definition of the zone_reset_threshold option
* Separated the patch to remove CHECK_SWD (or CHECK_VDB) to clarify the removal
Shin'ichiro Kawasaki (8):
zbd: refer file->last_start[] instead of sectors with data accounting
zbd: remove CHECK_SWD feature
zbd: rename the accounting 'sectors with data' to 'valid data bytes'
doc: fix unit of zone_reset_threshold and relation to other option
zbd: account valid data bytes only for zone_reset_threshold option
zbd: check write ranges for zone_reset_threshold option
zbd: initialize valid data bytes accounting at file setup
t/zbd: add test cases for zone_reset_threshold option
HOWTO.rst | 9 ++-
fio.1 | 8 ++-
t/zbd/test-zbd-support | 60 ++++++++++++++++-
zbd.c | 149 +++++++++++++++++++----------------------
zbd.h | 11 +--
5 files changed, 145 insertions(+), 92 deletions(-)
Applied. Thanks.
Vincent
