Given that hardware, buy something like this instead. used, but better class of card. https://www.newegg.com/intel-e10g42bt/p/N82E16833106075?Description=10gbit%20card&cm_re=10gbit_card-_-33-106-075-_-Product&quicklink=true or something similar from the used sellers. There seem to be a decent variety of cards under $100. Manufacturers that are good are HPE/DELL/Intel/Broadcom/IBM/Lenovo. But you do need to do a bit of research on the given cards to see what the real chipset is. Avoid Emulex/Be2net variants they have "issues". Intel/Broadcom/Mellanox based cards are good. On Fri, May 12, 2023 at 10:06 AM Thomas Cameron <thomas.cameron@xxxxxxxxxxxxxxx> wrote: > > On 5/12/23 09:41, Roger Heflin wrote: > > I have work experience with Intel 10Gbit, Older Emulex 10GB (be[23]net > > driver), current Broadcom 10/25G and Mellanox 10/25G. > > > > Note that for it to be useful you*MUST* have multiple interrupts. > > 1Gbit interfaces used to hit a limit at around 50Mbit (the cpu was not > > fast enough to do the required data moves on a single core to go much > > faster). Given the cpus are faster now the limit is probably up to > > 1Gbit/sec/interrupt. > > > > I have had to set nic adaptors to use a layer-3 data -> interrupt so > > that you can get higher rates (ethtool setting) only available on some > > nic cards. > > > > I have also had to set the interrupt number (ethtool setting), note > > that it is rather pointless to have the interrupt count higher than > > the number of real cores. And likely you want the interrupt count for > > the disk controllers being used + the nic interrupts <= the number of > > cores. > > > > At home I question the value of it. You might even simply test your > > nfs home setup and see if you can even get close to Gbit speeds, > > unless you have a newer machine and a good underlying disk setup you > > probably aren't going to get there. And you also need a decent Gbit > > card, but if you do not have a decent gbit card, you can buy a used > > enterprise grade dual gbit card for around $25 (branded Dell or > > branded HPE). > > > > With enterprise grade hw and good disk and other parts one can hit > > 115-125MBytes/sec. > > > > With a new machine at home and a 7-disk raid-6 setup with a good gbit > > nic card I can get 115-125MB/sec when data is in cache, but not coming > > directly off of disk, unless the disk is an ssd and reading large > > files. My prior machine could not hit that rate but was 10 year old > > hw. > > > > To use a 10Gbit interface you will have to have multiple machines > > doing large file sequential io (assuming they are wireless or gbit > > interfaces) at the same time. > > > > I doubt you are going to gain enough (or possibly any) to make it > > worth the cost or the trouble to get it working. > > > > Install sar and configure it to sample at 1minute and sar -n DEV will > > show you your network rates, if you aren't currently sustaining > > 115MByte/sec every so often then 10gbit is not going to do anything > > for you. > > My home office has a couple of HP Proliant servers with big (12 SAS > drives) RAID arrays and dual CPU E5-2697 v2 processors with 24 cores > each (48 cores total in the machine). I can sustain 1.6Gb/sec reads and > writes, tested via dd with oflag=dsync and also with fio. > > So with consumer grade NICs on the Proliants and my workstation > (https://www.amazon.com/TP-Link-TX401-Ethernet-Supports-Including/dp/B08D71PVXG/), > I suspect it will give me better performance than the 1Gb ethernet I'm > currently using. Will it be as amazing as high end enterprise gear? Most > likely not. Will it be better than 1Gb ethernet I'm currently using? I > suspect so. > > I appreciate the advice, I'll definitely play around with the settings > to make sure I'm using enough cores. > > Thomas > _______________________________________________ > users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue _______________________________________________ users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
