Both dual socket AMD Romes. Identical in every way. NUMAs per socket set to 1 in the BIOS. I'm using the exact same 10 drives on each system and they are PCIe Gen4 HPE OEM of SAMSUNG.... -----Original Message----- From: Jani Partanen <jiipee@xxxxxxxxxxx> Sent: Wednesday, January 26, 2022 3:32 PM To: Finlayson, James M CIV (USA) <james.m.finlayson4.civ@xxxxxxxx>; linux-raid@xxxxxxxxxxxxxxx Subject: [Non-DoD Source] Re: Showing my ignorance - kernel workers Hello, are both systems identical what comes to hardware? Mainly mobo. If no and they are dual socket systems, then it may be that one of the systems is designed to route all PCI-e via one socket so that all drive slots can be used just 1 socked populated. And another is designed so taht only half of the drive slots works when only 1 socket is populated. At least I have read something like this previously from this list. // JiiPee Finlayson, James M CIV (USA) kirjoitti 26/01/2022 klo 22.17: > I apologize in advance if you can point me to something I can read about mdraid besides the source code. I'm beyond the bounds of my understanding of Linux. Background, I do a bunch of NUMA aware computing. I have two systems configured identically with a NUMA node 0 focused RAID5 LUN containing NUMA node 0 nvme drives and a NUMA node 1 focused RAID5 LUN identically configured. 9+1 nvme, 128KB stripe, xfs sitting on top, 64KB O_DIRECT reads from the application.
