Ciao Matt, I'm definitely not a kernel expert, but i've been working a
lot with multicast lately.
I'll just tell you a couple of things I wish someone told me time ago.
- Multicast performance decreases with the number of senders.
I.e. if a single process can send data at 800Mbit/s, then two senders
(most likely) won't be able to send data at 400Mbit/s each.
(just saying that since you're using "many" servers)
- Timestamping after receive() can be very imprecise.
You should probably use in-kernel timestamping for that purpose.
Marco
On 5 Nov, 2009, at 16:15 , Matt Garman wrote:
Hello,
We are using multicast quite heavily for some internal messaging
across many Linux (CentOS 4.x, RHEL kernel 2.6.9) servers.
Effectively, we create our message, timestamp it (high resolution
timestamp via gettimeofday()), and call send(). The receiver gets
the message via recv(), adds another timestamp, and pushes it into a
queue. The queue is processed on a separate thread.
Note that all our machines are time sync'ed via NTP.
Part of that thread processing is analyzing the elapsed time between
send() and recv(). We are seeing increasing delays recently.
Though our traffic rate has increased, our machine load has not.
Our machines are all modern (Conroe or i7 architecture) dual-socket
quad core (eight total cores) Xeon boxes with 8 GB of RAM. The
machines' load is always under 1.00, and memory usage under 5%.
We are fairly certain the networking hardware is not the culprit, as
it is well under 50% load.
We *suspect* the Linux kernel scheduling may have something to do
with this. But this is only a hunch, and we do not know enough
about Linux internals to know where to being looking more closely.
Whenever I've done a net search for "linux multicast", the results
are always dated by several years, often talking about 2.4 and
earlier kernels. Given Linux's development pace, I'm afraid to put
much stock in this information.
So, a few questions:
- Does anyone have any experience dealing with issues similar to
what I described above?
- Can anyone point me to some recent information on Linux's
multicast implementation?
- Can anyone make some general comments about Linux, multicast,
scheduling, etc that might be relevant?
- I'm also looking for tests and/or benchmarks to help me ask a
more specific question.
I realize this is a somewhat vague question, but I'm at the "extreme
ignorance" point where I really don't know where to start or even
what questions to ask.
Any thoughts or suggestions would be most appreciated!
Thanks,
Matt
--
To unsubscribe from this list: send the line "unsubscribe linux-net"
in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
