On 2/8/2023 11:42 PM, Dave Hansen wrote: > On 2/8/23 10:03, Peter Zijlstra wrote: >>> - Hardware provided access information could be very useful for driving >>> hot page promotion in tiered memory systems. Need to check if this >>> requires different tuning/heuristics apart from what NUMA balancing >>> already does. >> I think Huang Ying looked at that from the Intel POV and I think the >> conclusion was that it doesn't really work out. What you need is >> frequency information, but the PMU doesn't really give you that. You >> need to process a *ton* of PMU data in-kernel. > > Yeah, there were two big problems. > > First, IIRC, Intel PEBS at the time only gave guest virtual addresses in > the PEBS records. They had to be translated back to host addresses to > be usable. That was extra expensive. Just to be clear, I am using IBS in host only and it can give both virtual and physical address. > > Second, it *did* take a lot of processing to turn raw memory accesses > into actionable frequency data. That meant that we started in a hole > performance-wise and had to make *REALLY* good decisions about page > migration to make up for it. I touched upon the frequency aspect in reply to Peter, but please let me know if I am missing something. > > The performance data here don't look awful, but they don't seem to add > up to a clear win either. I'm having a hard time imagining who would > turn this on and how widely it would get used in practice. I am hopeful with more appropriate tuning of NUMA balancing logic to work with hardware-provided access info (as against scan based NUMA hint faults), we should be able to see a clear win. At least theoretically we wouldn't have the overheads of address space scanning and hint faults handling. Thanks for your inputs. Regards, Bharata.
