Dear linux-net, I have recently taken over the development of the data acquisition system for a physics experiment. The first layer of this system needs to read data from custom electronics across a VME backplane at about 15 MB/s and forward it to a back-end machine for online analysis and storage. My predecessor invested lots of $$$ in a proprietary interconnection scheme that has proven to be extremely unstable in actual use. I am trying to replace it with gigabit Ethernet, but I am having trouble getting acceptable performance. The front-end processors are Motorola MVME 2600 boards, which have 200 MHz PowerPC 604e CPUs. I have equipped them with SBS Technologies PMC-Gigabit-ST3 daughtercards, which use the Intel 82545EM chipset. The PMC (PCI Mezzanine Connector--just PCI in a different form factor) slot is 66 MHz, 64-bit capable. I am running a 2.4.21-pre6 kernel from the linuxppc_2_4_devel BK tree, which includes version 4.4.19-k2 of the e1000 driver. The back-end server is a 1.7 GHz Pentium 4 with an Intel PRO/1000 T Server Adapter, and they are connected through a D-Link DGS-1008T gigabit switch. As measured with netperf and/or ttcp, I get between 80 and 90 Mbps from the front-end to the back-end server, which is a 1.7 GHz Pentium 4, even with the socket buffer size and TCP window size set to 256KB. In other words, I might as well have used the built-in 100BaseT port. The same benchmarks give around 350 Mbps between the server and another gigabit-equipped PC connected through the same switch, so I don't think my test is intrinsically flawed. While performing these benchmarks, the CPU utilization of netperf on the front-end is between 90% and 100%, suggesting that the process of sending the data is CPU-bound. I suppose that it might be occupied with computing the TCP checksum. The documentation for the 82545EM chipset claims that it supports transmit checksum offloading, but the Linux e1000 driver does not seem to use this feature, only receive checksum offloading. Is a 200 MHz PowerPC processor simply not adequate to drive a gigabit link faster than 80 Mbps, or do I have any options left to try? In our controlled environment, I could possibly consider hacking the kernel on both ends to disable checksum computation and verification, but I would certainly prefer not to have to resort to such extreme measures, and I am in any event not yet certain that the overhead comes from the checksum. I don't need full gigabit speed: 150 Mbps would be adequate. Thanks in advance to the networking gurus for their advice, -- Fred -- Fred Gray / Visiting Postdoctoral Researcher -- -- Department of Physics / University of California, Berkeley -- -- fegray@socrates.berkeley.edu / phone 510-642-4057 / fax 510-642-9811 -- - : send the line "unsubscribe linux-net" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
