From: K Prateek Nayak > Sent: 23 September 2022 16:38 .... > > This workaround is very painful on modern systems with a large number of > cores. The "inl()" can take thousands of cycles. Sampling certain > workloads with IBS on AMD Zen3 system shows that a significant amount of > time is spent in the dummy op, which incorrectly gets accounted as > C-State residency. A large C-State residency value can prime the cpuidle > governor to recommend a deeper C-State during the subsequent idle > instances, starting a vicious cycle, leading to performance degradation > on workloads that rapidly switch between busy and idle phases. > (For the extent of the performance degradation refer link [2]) Isn't that a horrid bug itself? Sounds like it affects any code that is doing pio reads of hardware buffers. While they are slow they are necessary. IIRC any PCIe read into an Altera fpga takes about 128 cycles of the 125MHz clock. The Intel cpu I've checked will only execute one concurrent PCIe read for each cpu core - so the cpu soon stalls for thousands of clocks. > The dummy wait is unnecessary on processors based on the Zen > microarchitecture (AMD family 17h+ and HYGON). Skip it to prevent > polluting the C-state residency information. Among the pre-family 17h > AMD processors, there has been at least one report of an AMD Athlon on a > VIA chipset (circa 2006) where this this problem was seen (see [3] for > report by Andreas Mohr). > > Modern Intel processors use MWAIT based C-States in the intel_idle driver > and are not impacted by this code path. For older Intel processors that > use the acpi_idle driver, a workaround was suggested by Dave Hansen and > Rafael J. Wysocki to regard all Intel chipsets using the IOPORT based > C-state management as being affected by this problem (see [4] for > workaround proposed). Can you use a surrogate (maybe AVX support?) to exclude large groups on modern cpu? Another possibility is that is the io address doesn't really matter are there any locations that have moved on-die and are now executed much faster than the ISA bus speed of older systems? Or do all the 'originally ISA' peripherals still run at ISA speeds? David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
