Hi, FYI, ltrace is very helpful for libgfapi clients. [root@hummingbird gfapi]# ltrace -c -ff -p 28906 ^C% time seconds usecs/call calls function ------ ----------- ----------- --------- -------------------- 43.27 5.952481 59524 100 glfs_rename 21.39 2.942866 29428 100 glfs_fsync 7.35 1.011715 10117 100 glfs_fsetxattr 7.26 0.999125 9991 100 glfs_creat 5.04 0.692812 1154 600 free 4.95 0.681092 851 800 __errno_location 4.67 0.641790 3208 200 glfs_stat 3.96 0.545340 908 600 malloc 1.26 0.173293 1732 100 glfs_close 0.84 0.115284 1152 100 glfs_write ------ ----------- ----------- --------- -------------------- 100.00 13.755798 2800 total Regards, -Prashanth Pai ----- Original Message ----- > From: "Pranith Kumar Karampuri" <pkarampu@xxxxxxxxxx> > To: "Gluster Devel" <gluster-devel@xxxxxxxxxxx> > Sent: Monday, January 4, 2016 10:49:50 AM > Subject: strace like utility for gluster > > hi, > I have been thinking of coming up with a utility similar to strace > for gluster. I am sending this mail to find out if anyone else is also > thinking about it and how they are solving this problem to see if we can > come up with a concrete solution that we can implement. > > These are my thoughts about the solution so far: > > 1) Doing trace for a brick/mount to know just what that process is > winding/unwinding (Or whatever else it wants to tell the trace process) > seems easier. We can launch a trace process which will open listener > unix socket to which the glusterfs process can send whatever it needs to > (glfstrace -h <brick-host> -p <brick-port> -s > <unix-socket-path-where-the-trace-needs-to-be-sent>). We will have to > write gf_trace() infra, which will do the job of sending the information > to this trace process only when there is a trace process trying to > listen in on what is happening inside the process. > > 2) Doing end-to-end tracing: > Unix socket based listening is not going to work anymore (Unless we send > the trace information in xdata or something, which doesn't seem nice to > me). We can use network sockets for sending the information from the > bricks to the trace process. So at the time of starting a trace on the > mount process, mount process will need to send an rpc to the brick > process giving the trace process hostname/port information for that > client, and bricks can send the trace information to the trace process > directly. So we can have multiple trace processes tracing different > mounts and bricks will be able to send trace information to different > trace processes. > > We will need to make sure gf_trace() is not going to send the > information to trace process in the io-path. > > 3) Doing glfstrace -p <application-pid>. I have an approach for fuse > based mounts. I am hoping nfs/smb folks will respond if we can do > something similar in those mounts. > > glfstrace process now traces the client process with extra information > i.e. frame->root->pid information in fuse-bridge which can be used to > filter only these fops executed by the application. Rest is similar to 2) > > 4) Doing glfstrace -c <shell-command> > > glfstrace process forks, it knows the pid of child, child should wait to > hear from parent to start 'exec-of-cmd'. glfstrace process sets things > up similar to 3), once it sets up tracing, tells child to exec the cmd. > > Comments are welcome :-). Happy new year by the way!! > > Pranith > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-devel > _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
