On Wed, 27 Aug 2014, Robert LeBlanc wrote: > I'm looking for a way to prioritize the heartbeat traffic higher than the > storage and replication traffic. I would like to keep the ceph.conf as > simple as possible by not adding the individual osd IP addresses and ports, > but it looks like the listening ports are pretty random. I'd like to use > iptables to mark the packet with DSCP, then our network team can mark it > with COS. I'm having a hard time figuring out what in the payload I can > match to say that it is a heartbeat packet. What would be best way for us to mark which sockets are heartbeat related? Is there some setsockopt() type call we should be using, or should we perhaps use a different port range for heartbeat traffic? > We have some concerns that under high network congestion scenarios, > heartbeats will be lost, marking OSDs down, triggering replication, causing > more traffic causing a compounding to the congestion, marking more OSDs down > until the entire cluster falls apart. Our intention is that we can try to > deliver heartbeat packets first with the hope that the system will know that > OSDs are up, just busy. Is our reasoning sound in this regard? It is reasonable. We do need to be careful, however. The heartbeats are attempting to measure whether the OSDs are able to effectively communicate. Every time we try to give heartbeats a different level of service, we are diminishing their ability to measure that reality. For example, if your QoS starved or nearly starved non-heartbeat traffic the OSDs wouldn't be able to tell. sage
 |