On 05/05/2016 01:36 PM, Mark Nelson wrote:
On 05/04/2016 03:51 PM, Jens Axboe wrote:
On 05/04/2016 11:43 AM, Mark Nelson wrote:
On 05/04/2016 11:34 AM, Jens Axboe wrote:
On 05/04/2016 08:10 AM, Jens Axboe wrote:
On 04/27/2016 02:38 PM, Jens Axboe wrote:
I'll try and see if I can reproduce this.
So I took a look at your logged output. The way that fio logs is
that it
does it on IO completion. You have set 1s intervals, so it checks,
when
an IO completes, if we're due for logging the previous interval IOPS:
spent = mtime_since(&td->iops_sample_time, t);
if (spent < td->o.iops_avg_time)
return;
If you are doing lots of IO, it matches pretty evenly. But for slow
workloads, it will end up being skewed a bit. It won't be too hard to
fix the drift, though, but it still won't make it clock precise at
exact
1s intervals.
We could move this part into the main thread, and out of the workers.
Then we could make it more precise. Might not be a bad idea in
general,
since it'll move some of this code out of the fast path.
I'll take a stab at it.
Can you try the below patch?
I'll also try to take a look at this as we've seen a fair amount of skew
in the logs doing fio librbd upstream ceph testing.
That'd be great. More complete version below, fwiw.
I actually wrote a
first stab at a parser to aggregate logs from multiple fio processes
with the assumption of non-uniform sampling:
https://github.com/ceph/cbt/blob/master/tools/fiologpaser.py
The idea is you have say 8 fio bw logs in a directory you can do:
fiologparser.py -f *bw*
which will return average values from the logs over user defined
intervals by taking weighted scores from samples that overlap. You can
also look at per-interval averages, sums, or just a global average.
various option flags:
-i <interval time in ms>
-d <divide results by d (say if you want a different unit of output)
-f <show full interval data>
-a <show average interval data across logs>
-s <show sums for interval data across logs>
That looks useful. Might want to put that in the fio repo as well, if
you are interested.
Sure, I'm game. Might be a little buggy and it's definitely slow since
I just threw it together the other day. Do you have a place in the repo
you like throwing parsing scripts?
Usually I put them in the tools/ directory.
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
