hi,
I have been thinking of coming up with a utility similar to strace
for gluster. I am sending this mail to find out if anyone else is also
thinking about it and how they are solving this problem to see if we can
come up with a concrete solution that we can implement.
These are my thoughts about the solution so far:
1) Doing trace for a brick/mount to know just what that process is
winding/unwinding (Or whatever else it wants to tell the trace process)
seems easier. We can launch a trace process which will open listener
unix socket to which the glusterfs process can send whatever it needs to
(glfstrace -h <brick-host> -p <brick-port> -s
<unix-socket-path-where-the-trace-needs-to-be-sent>). We will have to
write gf_trace() infra, which will do the job of sending the information
to this trace process only when there is a trace process trying to
listen in on what is happening inside the process.
2) Doing end-to-end tracing:
Unix socket based listening is not going to work anymore (Unless we send
the trace information in xdata or something, which doesn't seem nice to
me). We can use network sockets for sending the information from the
bricks to the trace process. So at the time of starting a trace on the
mount process, mount process will need to send an rpc to the brick
process giving the trace process hostname/port information for that
client, and bricks can send the trace information to the trace process
directly. So we can have multiple trace processes tracing different
mounts and bricks will be able to send trace information to different
trace processes.
We will need to make sure gf_trace() is not going to send the
information to trace process in the io-path.
3) Doing glfstrace -p <application-pid>. I have an approach for fuse
based mounts. I am hoping nfs/smb folks will respond if we can do
something similar in those mounts.
glfstrace process now traces the client process with extra information
i.e. frame->root->pid information in fuse-bridge which can be used to
filter only these fops executed by the application. Rest is similar to 2)
4) Doing glfstrace -c <shell-command>
glfstrace process forks, it knows the pid of child, child should wait to
hear from parent to start 'exec-of-cmd'. glfstrace process sets things
up similar to 3), once it sets up tracing, tells child to exec the cmd.
Comments are welcome :-). Happy new year by the way!!
Pranith
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
