Resurrecting a long-dead thread regarding multicast latencies. Original post is at the bottom. On Thu, Nov 05, 2009 at 05:02:25PM +0100, Marco P. wrote: > - Multicast performance decreases with the number of senders. > I.e. if a single process can send data at 800Mbit/s, then two senders > (most likely) won't be able to send data at 400Mbit/s each. > (just saying that since you're using "many" servers) That makes sense. However, we're sending lots of small packets, so it's more an issue of latency than throughput. > - Timestamping after receive() can be very imprecise. > You should probably use in-kernel timestamping for that purpose. How is this done? Is there a function call or a socket parameter? And how does one see the kernel timestamps to assess latencies? Thanks again, Matt > On 5 Nov, 2009, at 16:15 , Matt Garman wrote: > > >We are using multicast quite heavily for some internal messaging > >across many Linux (CentOS 4.x, RHEL kernel 2.6.9) servers. > > > >Effectively, we create our message, timestamp it (high resolution > >timestamp via gettimeofday()), and call send(). The receiver gets > >the message via recv(), adds another timestamp, and pushes it into a > >queue. The queue is processed on a separate thread. > > > >Note that all our machines are time sync'ed via NTP. > > > >Part of that thread processing is analyzing the elapsed time between > >send() and recv(). We are seeing increasing delays recently. > >Though our traffic rate has increased, our machine load has not. > >Our machines are all modern (Conroe or i7 architecture) dual-socket > >quad core (eight total cores) Xeon boxes with 8 GB of RAM. The > >machines' load is always under 1.00, and memory usage under 5%. > > > >We are fairly certain the networking hardware is not the culprit, as > >it is well under 50% load. > > > >We *suspect* the Linux kernel scheduling may have something to do > >with this. But this is only a hunch, and we do not know enough > >about Linux internals to know where to being looking more closely. > > > >Whenever I've done a net search for "linux multicast", the results > >are always dated by several years, often talking about 2.4 and > >earlier kernels. Given Linux's development pace, I'm afraid to put > >much stock in this information. > > > >So, a few questions: > > > > - Does anyone have any experience dealing with issues similar to > > what I described above? > > - Can anyone point me to some recent information on Linux's > > multicast implementation? > > - Can anyone make some general comments about Linux, multicast, > > scheduling, etc that might be relevant? > > - I'm also looking for tests and/or benchmarks to help me ask a > > more specific question. > > > >I realize this is a somewhat vague question, but I'm at the "extreme > >ignorance" point where I really don't know where to start or even > >what questions to ask. > > > >Any thoughts or suggestions would be most appreciated! -- To unsubscribe from this list: send the line "unsubscribe linux-net" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
