On Tue, Jan 24, 2017 at 10:39 AM, Vijay Bellur <vbellur@xxxxxxxxxx> wrote: > > > On Thu, Jan 19, 2017 at 8:06 AM, Jeff Darcy <jdarcy@xxxxxxxxxx> wrote: >> >> > The more relevant question would be with TCP_KEEPALIVE and >> > TCP_USER_TIMEOUT >> > on sockets, do we really need ping-pong framework in Clients? We might >> > need >> > that in transport/rdma setups, but my question is concentrating on >> > transport/rdma. In other words would like to hear why do we need >> > heart-beat >> > mechanism in the first place. One scenario might be a healthy socket >> > level >> > connection but an unhealthy brick/client (like a deadlocked one). >> >> This is an important case to consider. On the one hand, I think it >> answers >> your question about TCP_KEEPALIVE. What we really care about is whether a >> brick's request queue is moving. In other words, what's the time since >> the >> last reply from that brick, and does that time exceed some threshold? On >> a >> busy system, we don't even need ping packets to know that. We can just >> use >> responses on other requests to set/reset that timer. We only need to send >> ping packets when our *outbound* queue has remained empty for some >> fraction >> of our timeout. >> >> However, it's important that our measurements be *end to end* and not just >> at the transport level. This is particularly true with multiplexing, >> where multiple bricks will share and contend on various resources. We >> should ping *through* client and server, with separate translators above >> and below each. This would give us a true end-to-end ping *for that >> brick*, and also keep the code nicely modular. > > > +1 to this. Having ping, pong xlators immediately above and below protocol > translators would also address the problem of epoll threads getting blocked > in gluster's xlator stacks in busy systems. IMO, as Jeff mentioned for ping timeout that it is required only when there are no operations happening, similarly, we can consider frame-timeout as a mechanism of *end to end* keep alive. What extra feature(s) do we get by adding a ping-pong xlator couple? > > Having said that, I do see value in Rafi's patch that prompted this thread. > Would it not help to prioritize ping - pong traffic in all parts of the > gluster stack including the send queue on the client? > > Regards, > Vijay _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-devel
