Hi all, Ceph relies on TCP for inter-node communication. One thing that might be interesting is to use a cluster-optimized network transport protocol for Ceph. For example, TIPC provides the same sockets interface, and can provide reliable delivery. But unlike TCP, TIPC was optimized for cluster environments where packet loss is low, bandwidth is high, etc. Ericsson uses it for cluster applications. (See http://tipc.sourceforge.net/ ) There are some papers out there that benchmark TCP vs. TIPC and other protocols. Basically TCP has a lot of mechanisms that were created to work well on the open internet, which is kind of a risky place. But dropping them can result in better latency and bandwidth in a closed cluster. TIPC uses a different acknowledgement mechanism, doesn't checksum packets since it assumes that the hardware does that, avoids the overhead of the 3-way handshake, etc. There is apparently a native interface and a sockets interface to TIPC. I'm not sure what the advantages of the native interface are versus sockets. Ideally this would be a drop-in replacement. Just a random idea. cheers, Colin McCabe -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
