Avati that's awesome! I didn't know that the meta xlator could do that already.
On Sat, May 17, 2014 at 9:30 AM, Anand Avati <avati@xxxxxxxxxxx> wrote:
KP, Vipul,It will be awesome to get io-stats like instrumentation on the client side. Here are some further thoughts on how to implement that. If you have a recent git HEAD build, I would suggest that you explore the latency stats on the client side exposed through meta at $MNT/.meta/graphs/active/$xlator/profile. You can enable latency measurement with "echo 1 > $MNT/.meta/measure_latency". I would suggest extending these stats with the extra ones io-stats has, and make glusterfsiostats expose these stats.If you can compare libglusterfs/src/latency.c:gf_latency_begin(), gf_latency_end() and gf_latency_udpate() with the macros in io-stats.c UPDATE_PROFILE_STATS() and START_FOP_LATENCY(), you will quickly realize how a lot of logic is duplicated between io-stats and latency.c. If you can enhance latency.c and make it capture the remaining stats what io-stats is capturing, the benefits of this approach would be:- stats are already getting captured at all xlator levels, and not just at the position where io-stats is inserted- file like interface makes the stats more easily inspectable and consumable, and updated on the fly- conforms with the way rest of the internals are exposed through $MNT/.metaIn order to this, you might want to look into:- latency.c as of today captures fop count, mean latency, total time, whereas io-stats measures these along with min-time, max-time and block-size histogram.- extend gf_proc_dump_latency_info() to dump the new stats- either prettify that output like 'volume profile info' output, or JSONify it like xlators/meta/src/frames-file.c- add support for cumulative vs interval stats (store an extra copy of this->latencies[])
etc..Thanks!On Fri, Apr 25, 2014 at 9:09 PM, Krishnan Parthasarathi <kparthas@xxxxxxxxxx> wrote:
[Resending due to gluster-devel mailing list issue]
Apologies for the late reply.
glusterd uses its socket connection with brick processes (where io-stats xlator is loaded) to
gather information from io-stats via an RPC request. This facility is restricted to brick processes
as it stands today.
Some background ...
io-stats xlator is loaded, both in GlusterFS mounts and brick processes. So, we have the capabilities
to monitor I/O statistics on both sides. To collect I/O statistics at the server side, we have
# gluster volume profile VOLNAME [start | info | stop]
AND
#gluster volume top VOLNAME info [and other options]
We don't have a usable way of gathering I/O statistics (not monitoring, though the counters could be enhanced)
at the client-side, ie. for a given mount point. This is the gap glusterfsiostat aims to fill. We need to remember
that the machines hosting GlusterFS mounts may not have glusterd installed on them.
We are considering rrdtool as a possible statistics database because it seems like a natural choice for storing time-series
data. rrdtool is capable of answering high-level statistical queries on statistics that were logged in it by io-stats xlator
over and above printing running counters periodically.
Hope this gives some more clarity on what we are thinking.
thanks,
Krish
----- Original Message -----
> Probably me not understanding.
> the comment "iostats making data available to glusterd over RPC" - is what I
> latched on to. I wondered whether this meant that a socket could be opened
> that way to get at the iostats data flow.
> Cheers,
> PC
> ----- Original Message -----
> > From: "Vipul Nayyar" <nayyar_vipul@xxxxxxxxx>
>
> > To: "Paul Cuzner" <pcuzner@xxxxxxxxxx>, "Krishnan Parthasarathi"
> > <kparthas@xxxxxxxxxx>
>
> > Cc: "Vijay Bellur" <vbellur@xxxxxxxxxx>, "gluster-devel"
> > <gluster-devel@xxxxxxxxxx>
>
> > Sent: Thursday, 20 February, 2014 5:06:27 AM
>
> > Subject: Re: New project on the Forge - gstatus
>
> > Hi Paul,
>
> > I'm really not sure, if this can be done in python(at least comfortably).
> > Maybe we can tread on the same path as Justin's glusterflow in python. But
> > I
> > don't think, all the io-stats counters will be available with the way how
> > Justin's used Jeff Darcy's previous work to build his tool. I can be wrong.
> > My knowledge is a bit incomplete and based on very less experience as a
> > user
> > and an amateur Gluster developer. Please do correct me, if I can be.
>
> > Regards
>
> > Vipul Nayyar
>
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-devel
