On 20 March 2018 at 05:57, Kris Davis <Kris.Davis@xxxxxxx> wrote:
> I created a pull request from github with the latest changes, both to fiologparser_hist.py and the associated man page. This includes a new "--directions=rwtm" option to allow independent directional results to be printed (with an added 'dir' column).
I see Jens pulled it in so congrats on staying the course!
> If there is a way to infer whether the histogram results are in 'ns' or 'ms' from the number of columns, please let me know. I wonder if the 'coarseness' option might be a factor they might complicate things. I'm mostly relying on the original script operation, rather than having a more complete understanding of the dependencies.
The coarseness definitely complicates things but here are my thoughts:
If you know the total columns and you're willing to assume fio compile
defaults you can try and guess whether FIO_IO_U_PLAT_GROUP_NR is 19
(in which case you assume usec) or 29 (in which case you assume nsec).
>From iolog.c:
723 int stride = 1 << hist_coarseness;
[...]
734 s = __get_sample(samples, log_offset, i);
735
736 entry = s->data.plat_entry;
737 io_u_plat = entry->io_u_plat;
738
739 entry_before = flist_first_entry(&entry->list,
struct io_u_plat_entry, list);
740 io_u_plat_before = entry_before->io_u_plat;
741
742 fprintf(f, "%lu, %u, %u, ", (unsigned long) s->time,
743 io_sample_ddir(s), s->bs);
744 for (j = 0; j < FIO_IO_U_PLAT_NR - stride; j += stride) {
745 fprintf(f, "%llu, ", (unsigned long long)
746 hist_sum(j, stride, io_u_plat,
io_u_plat_before));
747 }
748 fprintf(f, "%llu\n", (unsigned long long)
749 hist_sum(FIO_IO_U_PLAT_NR - stride,
stride, io_u_plat,
750 io_u_plat_before));
So we can tell the number of columns is related to the 3 +
(FIO_IO_U_PLAT_NR / (2 ^ coarseness)) + 1. The 3 comes from printing
the time, the data direction and the blocksize columns. In python
def main():
for pgn in [19, 29]:
print("pgn=%d:" % (pgn)),
for c in range(7):
pn = (1 << 6) * pgn
s = (1 << c)
print(((pn - s) / s) + 4),
print("")
if __name__ == "__main__":
main()
prints the following:
pgn=19: 1219 611 307 155 79 41 22
pgn=29: 1859 931 467 235 119 61 32
Quick comparison:
$ ./fio --version
fio-3.5-70-g40f1
$ ./fio --name=test --rw=read --runtime=200ms --time_based
--filename=/tmp/fio.tmp --size=1M --write_hist_log=test
--log_hist_msec=100 --log_hist_coarseness=0
$ head -1 test_clat_hist.1.log | awk -F',' '{print NF}'
1859
./fio --name=test --rw=read --runtime=200ms --time_based
--filename=/tmp/fio.tmp --size=1M --write_hist_log=test
--log_hist_msec=100 --log_hist_coarseness=6
$ head -1 test_clat_hist.1.log | awk -F',' '{print NF}'
32
If we approximate the equation we get
((64 * FIO_IO_U_PLAT_GROUP_NR) / (2 ^ coarseness)) + 4 = total_columns;
which can be rearranged to
(64 * FIO_IO_U_PLAT_GROUP_NR) / (total_columns - 4) = (2 ^ coarseness)
log2((64 * FIO_IO_U_PLAT_GROUP_NR) / (total_columns - 4) = coarseness
This allows us to guess and check a bit quicker. For example with
FIO_IO_U_PLAT_GROUP_NR=19, columns=611
import math
def check_columns(column_actual, pgn_guess):
approx_coarse_guess = int(math.log((64 * pgn_guess) /
float(column_actual - 4), 2))
stride_guess = 1 << approx_coarse_guess
column_guess = ((64 * pgn_guess - stride_guess) / stride_guess) + 4
return column_guess == column_actual
>>> check_columns(611, 19)
True
A bigger test:
>>> [check_columns(x, 19) for x in (1219, 611, 307, 155, 79, 41, 22)]
[True, True, True, True, True, True, True]
>>> [check_columns(x, 19) for x in (1859, 931, 467, 235, 119, 61, 32)]
[False, False, False, False, False, False, False]
If someone explicitly sets --group_nr they ought to set whether the
time is in usec or nsec (or generally override the default) as we
can't really guess any more.
What do you think?
--
Sitsofe | http://sucs.org/~sits/
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
