On 30.11.21 12:51, liupeng (DM) wrote: > Hi David, > > Hi! > > We met a performance regression(lmbench bw_mmap_rd/bw_mem) on our server > with > > patch 7fef431be9c9(mm/page_alloc: place pages to tail in > __free_pages_core()), > > > > Case A. ./bw_mmap_rd -P 1 512m mmap_only out.file > > Case B. ./bw_mem -P 1 512m frd > > > > mode0: IMC Interleave = auto, Channel Interleave = auto, Rank Interleave > = auto > > without with regression > > case A 10535.14 9645.02 8.4% > > case B 10526.88 9797.63 6.9% > > also we found different memory interleaving have different results > > > > mode1: IMC Interleave = 1-way, Channel Interleave = 2-way, Rank > Interleave =2-way > > without with > regression > > case A 10543.01 10643.12 -0.9% > > case B 10971.33 10853.99 1% > > > > The 7fef431be9c9 changes the different memory layout, it seems that the > issue It doesn't change the physical memory layout. It changes the order in which physical memory will get allocated. > > is related with memory interleave. But the patch does lead to > performance regression. > > > > Any suggestion about this regression? Does it happen only on specific HW? Commit 7fef431be9c9 discovered a bunch of BUGs already, including physical memory that has different performance characteristics due to firmware bugs, which indicated some physical memory e.g., as uncached to the OS. That commit triggers allocation of physical memory in a different order, which sometimes makes "bad physical memory" getting allocated earlier than later in some benchmarks. Such effects are usually only visible on a freshly started system; once the system was busy for some time and the free page lists are nondeterministic, you might observe similar results. I wrote a kernel module that was able to identify bad physical memory regions in some affected setups: https://github.com/davidhildenbrand/kstream/blob/main/README.md You can try running it after a freshly booted system to eventually identify some "bad" physical memory areas. But it might not be the root cause. Maybe it's just some caching effects? -- Thanks, David / dhildenb
