Re: multicast performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Resurrecting a long-dead thread regarding multicast latencies.
Original post is at the bottom.

On Thu, Nov 05, 2009 at 05:02:25PM +0100, Marco P. wrote:
> - Multicast performance decreases with the number of senders.
> I.e. if a single process can send data at 800Mbit/s, then two senders  
> (most likely) won't be able to send data at 400Mbit/s each.
> (just saying that since you're using "many" servers)

That makes sense.  However, we're sending lots of small packets, so
it's more an issue of latency than throughput.

> - Timestamping after receive() can be very imprecise.
> You should probably use in-kernel timestamping for that purpose.

How is this done?  Is there a function call or a socket parameter?
And how does one see the kernel timestamps to assess latencies?

Thanks again,
Matt


> On 5 Nov, 2009, at 16:15 , Matt Garman wrote:
>
> >We are using multicast quite heavily for some internal messaging
> >across many Linux (CentOS 4.x, RHEL kernel 2.6.9) servers.
> >
> >Effectively, we create our message, timestamp it (high resolution
> >timestamp via gettimeofday()), and call send().  The receiver gets
> >the message via recv(), adds another timestamp, and pushes it into a
> >queue.  The queue is processed on a separate thread.
> >
> >Note that all our machines are time sync'ed via NTP.
> >
> >Part of that thread processing is analyzing the elapsed time between
> >send() and recv().  We are seeing increasing delays recently.
> >Though our traffic rate has increased, our machine load has not.
> >Our machines are all modern (Conroe or i7 architecture) dual-socket
> >quad core (eight total cores) Xeon boxes with 8 GB of RAM.  The
> >machines' load is always under 1.00, and memory usage under 5%.
> >
> >We are fairly certain the networking hardware is not the culprit, as
> >it is well under 50% load.
> >
> >We *suspect* the Linux kernel scheduling may have something to do
> >with this.  But this is only a hunch, and we do not know enough
> >about Linux internals to know where to being looking more closely.
> >
> >Whenever I've done a net search for "linux multicast", the results
> >are always dated by several years, often talking about 2.4 and
> >earlier kernels.  Given Linux's development pace, I'm afraid to put
> >much stock in this information.
> >
> >So, a few questions:
> >
> >   - Does anyone have any experience dealing with issues similar to
> >     what I described above?
> >   - Can anyone point me to some recent information on Linux's
> >     multicast implementation?
> >   - Can anyone make some general comments about Linux, multicast,
> >     scheduling, etc that might be relevant?
> >   - I'm also looking for tests and/or benchmarks to help me ask a
> >     more specific question.
> >
> >I realize this is a somewhat vague question, but I'm at the "extreme
> >ignorance" point where I really don't know where to start or even
> >what questions to ask.
> >
> >Any thoughts or suggestions would be most appreciated!
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Netdev]     [Ethernet Bridging]     [Linux 802.1Q VLAN]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Git]     [Bugtraq]     [Yosemite News and Information]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux PCI]     [Linux Admin]     [Samba]

  Powered by Linux