Hello,
Basing my first patch to Gluster as a stepping stone, I've written a small utility
glusterfsiostat, in python which you can find attached with this email. Currently, the modifications done by my patch to io-stats which is under review as of now, dumps private information from the xlator object to the proper file for private info in the meta directory. This includes total bytes read/written along with read/write speed in the previous 10 seconds. The speed at every 1 second is identified by it's respective unix timestamp and hence given out in bytes/second. These values at discrete points of time can be used to generate a graph.
The python tool first identifies all gluster mounts in the system, identifies the mount path and parses the meta xlator output in order to generate output similar to the iostat tool. Passing '-j' option gives you extra information in a consumable json format. By default, the
tool pretty prints the basic stats which are human readable. This tool is supposed to be a framework on which other applications can be built upon. I'm putting this out for community feedback so as to improve it further.
Do you think the stats and the way they're generated as of now is something that might be usable by someone? Any other implementation suggestions or addition of some more stats that I can/should possibly provide with the utility?
Note: In order to test this, you need to apply my patch(http://review.gluster.org/#/c/8030/) in your repo first, build and then mount a volume. Preferably perform a big read/write operation with a file on your Gluster mount before executing the python script. Then run it as 'python stat.py' or 'python stat.py -j'
Also, please try to answer my question on similar lines, posted here : http://stackoverflow.com/questions/24332368/will-displaying-an-immediate-i-o-speed-be-a-better-alternative-than-showing-an-a
Regards
Vipul Nayyar
# glusterfsiostat - A client side tool to gather I/O stats from every Gluster mount.
# Author : Vipul Nayyar <nayyar_vipul@xxxxxxxxx>
import commands
import re
import os
import json
import sys
from optparse import OptionParser
status, output=commands.getstatusoutput('mount')
mountlines = output.split("\n")
if status != 0:
print "Unable to gather mount statistics"
exit(1)
mntarr = []
for i in mountlines:
matchobj = re.search(r" type fuse.glusterfs \(.*\)$",i)
if matchobj:
i = i.replace(matchobj.group(),"")
i = i.split(" on ")
mntname = i[0]
mntpath = i[1]
temp = {}
temp["mount_path"] = mntpath
temp["name"] = mntname
mntarr.append(temp)
for i in xrange(0,len(mntarr)):
os.chdir(mntarr[i]["mount_path"])
os.chdir(".meta")
os.chdir("graphs/active")
status, output=commands.getstatusoutput("ls")
if status != 0:
print mntpath + ": components not accessible"
continue
lsarr = output.split('\n')
for j in lsarr:
io_stats_path = ""
status, output=commands.getstatusoutput("cat "+ j + "/type")
if output == "debug/io-stats":
os.chdir(j)
io_stats_path = os.getcwd()
break
if io_stats_path == "": continue
priv_content = commands.getstatusoutput("cat private")
priv_content = priv_content[1].split('\n')
mntarr[i]["read_speed"] = {}
mntarr[i]["write_speed"] = {}
for j in priv_content:
match = re.search(r"write_speed\((.*)\) = (.*)$",j)
if(match):
mntarr[i]["read_speed"][match.group(1)] = match.group(2)
match = re.search(r"read_speed\((.*)\) = (.*)$",j)
if(match):
mntarr[i]["write_speed"][match.group(1)] = match.group(2)
match = re.search(r"data_read_cumulative = (.*)$",j)
if(match):
mntarr[i]["read_cumulative"] = match.group(1)
match = re.search(r"data_read_incremental = (.*)$",j)
if(match):
mntarr[i]["read_incremental"] = match.group(1)
match = re.search(r"data_written_cumulative = (.*)$",j)
if(match):
mntarr[i]["write_cumulative"] = match.group(1)
match = re.search(r"data_written_incremental = (.*)$",j)
if(match):
mntarr[i]["write_incremental"] = match.group(1)
parser = OptionParser()
parser.add_option("-j", "--json", action = "store_true", dest = "json", help = "Get extra output in json format", default = False)
(options, args) = parser.parse_args()
if(len(mntarr) == 0):
print "No gluster mounts found."
exit(1)
if(options.json == True):
print json.dumps(mntarr)
else:
sys.stdout.write("Device:")
length = len(mntarr[i]["name"]) - 7
if(length > 0):
for x in xrange(0,length):
sys.stdout.write(" ")
sys.stdout.write(" kB_read/s kB_wrtn/s kB_read kB_wrtn\n")
for i in xrange(0,len(mntarr)):
if(len(mntarr[i]["read_speed"]) > 0):
r_speed = int(mntarr[i]["read_speed"][max(mntarr[i]["read_speed"].iterkeys())])/1024
else:
r_speed = 0
if(len(mntarr[i]["write_speed"]) > 0):
w_speed = int(mntarr[i]["write_speed"][max(mntarr[i]["write_speed"].iterkeys())])/1024
else:
w_speed = 0
sys.stdout.write(str(mntarr[i]["name"])+" " + str(r_speed) + " " + str(w_speed) + " ")
sys.stdout.write(str(int(mntarr[i]["read_cumulative"])/1024) + " ")
sys.stdout.write(str(int(mntarr[i]["write_cumulative"])/1024))
sys.stdout.write("\n")
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
