From: Niklas Cassel <niklas.cassel@xxxxxxx> While the regular latency stats support collecting data for clat, slat and lat (clat+slat) at the same time, the per priority latencies can only track either clat or lat, and only for two different priorities (high/low). This patch series introduces support for a dynamically allocated array of priorities (capped at max 64 different prios per ddir), while still being limited to only track one latency type at a time. This patch series also introduces a new extended input format for the cmdprio_bssplit option, which allows the user to specify a number of blocksize/percentage/priorityclass/prioritylevel entries per ddir. The output has been modified to print the stats for each priority class and level combination. This will only be printed if cmdprio_percentage or cmdprio_bssplit is used, or if multiple jobs (in the same group) specified different priorities using prioclass/prio options. NOTE: The json output will no longer contain high_prio/low_prio entries, but will instead include a new list "prios", which will include an object per prioclass/priolevel combination. Each of these objects will either have a "clat_ns" object or a "lat_ns" object, depending on which latency type was being tracked. This json structure should make it easy if the per priority stats were ever extended to be able to track multiple latency types at the same time, as each prioclass/priolevel object will then simply contain (e.g.) both a "clat_ns" and a "lat_ns" object. The code can also be cloned using git: git clone -b multi-prio-v2 https://github.com/floatious/fio.git Kind regards, Niklas Changes since v1: -Fixed review comments by Damien. -Picked up tags from Damien. -Added a new patch (6/18). Niklas Cassel (18): init: verify option lat_percentiles consistency for all jobs in group backend: do ioprio_set() before calling the ioengine init callback stat: save the default ioprio in struct thread_stat client/server: convert ss_data to use an offset instead of fixed position stat: add a new function to allocate a clat_prio_stat array os: define min/max prio class and level for systems without ioprio options: add a parsing function for an additional cmdprio_bssplit format cmdprio: add support for a new cmdprio_bssplit entry format examples: add new cmdprio_bssplit format examples stat: use enum fio_ddir consistently stat: increment members counter after call to sum_thread_stats() stat: add helper for resetting the latency buckets stat: disable per prio stats where not needed stat: report clat stats on a per priority granularity stat: convert json output to a new per priority granularity format gfio: drop support for high/low priority latency results stat: remove unused high/low prio struct members t/latency_percentiles.py: add tests for the new cmdprio_bssplit format HOWTO | 26 +- backend.c | 25 +- client.c | 43 ++- engines/cmdprio.c | 440 ++++++++++++++++++++++++----- engines/cmdprio.h | 22 +- engines/filecreate.c | 2 +- engines/filedelete.c | 2 +- engines/filestat.c | 2 +- examples/cmdprio-bssplit.fio | 39 ++- fio.1 | 32 ++- gclient.c | 55 +--- init.c | 24 ++ io_u.c | 7 +- io_u.h | 3 +- options.c | 122 ++++++++ os/os.h | 4 + rate-submit.c | 9 + server.c | 99 +++++-- server.h | 2 +- stat.c | 529 +++++++++++++++++++++++++++-------- stat.h | 40 ++- t/latency_percentiles.py | 211 +++++++++----- thread_options.h | 10 + 23 files changed, 1361 insertions(+), 387 deletions(-) -- 2.34.1
