On Sat, Jul 25, 2015 at 5:12 AM, chris holcombe <chris.holcombe@xxxxxxxxxxxxx> wrote:
Hey Everyone!
I'm having trouble figuring out how the RPC mechanism works in Gluster. Any help would be hugely appreciated!
When a client makes a request from the glusterd service an xdr encoded rpc is sent to it. From what I'm gathering by looking at the source and GDB the main struct to be concerned with is:
struct rpc_transport_req {
rpc_transport_msg_t msg;
rpc_transport_rsp_t rsp;
struct rpc_req *rpc_req;
};
>From tracing it down to writev I see that the data looks as follows before being written out:
(gdb) p req
$4 = {msg = {rpchdr = 0x7ffff36c7c00, rpchdrcount = 1, proghdr = 0x7ffff36c7d20,
proghdrcount = 1, progpayload = 0x0, progpayloadcount = 0, iobref = 0x7fffe4000920}, rsp = {
rsphdr = 0x0, rsphdr_count = 0, rsp_payload = 0x0, rsp_payload_count = 0, rsp_iobref = 0x0},
rpc_req = 0x555555802dac}
I'm having trouble figuring out if this req variable is actually what is being written out to the wire.
Now on the reply side the first header is the length of the reply. That makes sense. What I'm not getting is if the rpc_transport_req is actually being serialized with xdr or are parts of it being serialized? It's hard to tell. I need to read the __socket_proto_state_machine over again and understand it better. I searched around for notes or anything online and there's really almost zero written about this.
So my questions are:
1. Is the rpc_transport_req being xdr encoded onto the wire?
No, its the contents of rpc_transport_msg_t (req->msg, defined in rpc-transport.h) which is written onto the wire. For more details look into functions __socket_ioq_new and __socket_ioq_churn_entry (both defined in ./rpc/rpc-transport/socket/src/socket.c).
2. Are there any good materials out there that explain the protocol? Figuring it out by reading the source and break pointing the code is slow going. Esp with the epoll threads making tracking things down hard.
Our rpc implementation is based on RFC 5531 [1]. To get an high level flow, you can look at socket_submit_request and socket_submit_reply and keep following the control flow.
[1] https://tools.ietf.org/html/rfc5531
[1] https://tools.ietf.org/html/rfc5531
regards,
Raghavendra.
Thanks for the help!
-Chris
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
--
Raghavendra G
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
