I have a Sun T1000 with a T1 processor (8 cores, 4 strands per core). The machine is
equipped with two BCM5714 and two BCM5407 Gigabit NICs. I am using kernel 2.6.26.5, with
version 3.92.1 of the tg3 driver.
The problem is that I cannot get peak TCP throughput at both directions at the same time.
The experiment I'm running is quite simple:
1. Run a TCP server that reads and discards data on the T1000
2. Run the same server on a dual Xeon machine
3. Start sending data from the Xeon machine to the T1000
4. Start sending data from the T1000 machine to the Xeon
If I skip stage 3, than the T1000 can transmit at 1Gbps. Similarly, if I skip stage 4, the
T1000 can receive at 1Gbps. However, when transmitting and receiving at the same time, one
of the directions drops to around 200 Mbps.
A couple of notes:
1. Since each strand on the T1 is quite weak, interrupt handling, transmission and
reception are spread across different strands. The best performance is obtained when these
are spread across cores.
2. According to /proc/stat, there is considerable idle time on all strands when the
machine both transmits and receives, so that the experiment does not seem to be CPU bound
3. If transmission and reception are performed in the exact same manner, only using two
separate NICs, all is well (again, hinting that the experiment is not CPU bound).
I don't have any experience with these Broadcom NICs, but I assume that they support
duplex Gigabit Ethernet. I suspect a driver problem at this point, but I'm open to
suggestions.
Thanks,
Elad
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
