Thanks Mark, we did look at tools from Ben England here https://github.com/bengland2/cbt/blob/fio-thru-analysis/tools/parse-fio-result-log.sh but not with much luck, that’s partly because we didn’t bother to look into the gory details if things didn’t work. Thanks for the scripts you have attached, will give a shot. Thanks, -Pavan. On 7/6/16, 11:05 PM, "ceph-users on behalf of Mark Nelson" <ceph-users-bounces@xxxxxxxxxxxxxx on behalf of mnelson@xxxxxxxxxx> wrote: Hi Pavan, A couple of us have some pretty ugly home-grown scripts for doing this. Basically just bash/awk that loop through the directories and grab the fio bw/latency lines. Eventually the whole way that cbt stores data should be revamped since the way data gets laid out in a nested directory structure doesn't really scale. Lately we've been more focused on parsing the fio log output. We've had issues with strange clock skew and multiple concurrent clients not all running at exactly the same time, leading to bias in the results. To get around this, we wrote a tool to parse the output of multiple fio bw/iops/latency log files. It was upstreamed into fio itself recently here: https://github.com/axboe/fio/blob/master/tools/fiologparser.py a potentially better (but still experimental and maybe buggy) version of this is here: https://github.com/markhpc/fio/blob/wip-interval/tools/fiologparser.py The idea is to fit samples into intervals based on how much they overlap, but if you have multiple files (say from multiple simulatenous clients) only calculate averages based on the time the clients ran at the same time (in case some clients started late or ended early). Ben England is working on speeding this up when fio records every IO rather than interval averages and we have an intern now that is looking at this and other methods for getting better logging data out of fio. Anyway, I've included two scripts that will run through a cbt output directory and either read the fio output file or run this script to parse the output data. Like I said, these are basically terrible (even embarrassing!) and will almost certainly need some slight hacking to work based on the iodepth/readahead/etc. I think Ben England has something a bit nicer but he's on vacation at the moment so you'll have to suffer with my horrible hacked up bash for now. ;) Mark On 07/06/2016 03:22 AM, Pavan Rallabhandi wrote: > Wanted to check if there are any readily available tools that the community is aware of/using for parsing/plotting CBT run results. Am particularly interested in tools for the CBT librbdfio runs, where in the aggregated BW/IOPS/Latency reports are generated either in CSV/graphical. > > Thanks! > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |