Not logging zero entry intervals when using --log_avg_msec seems problematic

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I was digging into the bandwidth/iops logging code in stat.c after
thinking about how to compute an average bandwidth/iops over a period
of time given interval log entries.

I noticed something concerning. When there is an interval where
nothing happened (e.g. no read iops), nothing is logged but the time
of last interval is advanced (there is only one such time for each
log, shared by all directions).

This makes sense for a case when you are only doing reads and don't
want write entries in your log. But it means that looking at a log
file can be pretty misleading. If you are logging with an interval of
100ms but only one read is occurring per second, you get a log for
iops like this:

149, 6, 0, 0, 0
1056, 10, 0, 0, 0
2064, 9, 0, 0, 0
3072, 10, 0, 0, 0
4081, 10, 0, 0, 0

If you turn the interval up to every 10 ms, you get this:
149, 6, 0, 0, 0
1011, 100, 0, 0, 0
2011, 100, 0, 0, 0
3011, 100, 0, 0, 0
4011, 100, 0, 0, 0

Those zero entries are actually really important in a case like this.
Those logs do not look like logs for a job that did 1 read per second.
This is an extreme example to show the problem, and of course we know
what's going on here.

But what if you're doing a practical test on storage with a fairly
small log interval, and the storage hiccups and processes no
operations in an interval? It can be impossible to know if an apparent
"gap" in a log file is because there was a zero entry or a system
hiccup caused the logging process to run a little late.

Any thoughts? Just logging the zero entries would work, but at the
expense of noise in logs.

(One option I can think of is remembering, per direction, that the
last interval was a zero-entry one, and then writing that if a
non-zero interval is later encountered. Also the last entry per
direction must always be written, if any non-zero interval for that
direction was ever written.)

Thanks,
Nick



[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux