Bharata B Rao <bharata@xxxxxxx> writes: > On 2/13/2023 8:26 AM, Huang, Ying wrote: >> Bharata B Rao <bharata@xxxxxxx> writes: >> >>> On 2/8/2023 11:33 PM, Peter Zijlstra wrote: >>>> On Wed, Feb 08, 2023 at 01:05:28PM +0530, Bharata B Rao wrote: >>>> >>>> >>>>> - Hardware provided access information could be very useful for driving >>>>> hot page promotion in tiered memory systems. Need to check if this >>>>> requires different tuning/heuristics apart from what NUMA balancing >>>>> already does. >>>> >>>> I think Huang Ying looked at that from the Intel POV and I think the >>>> conclusion was that it doesn't really work out. What you need is >>>> frequency information, but the PMU doesn't really give you that. You >>>> need to process a *ton* of PMU data in-kernel. >>> >>> What I am doing here is to feed the access data into NUMA balancing which >>> already has the logic to aggregate that at task and numa group level and >>> decide if that access is actionable in terms of migrating the page. In this >>> context, I am not sure about the frequency information that you and Dave >>> are mentioning. AFAIU, existing NUMA balancing takes care of taking >>> action, IBS becomes an alternative source of access information to NUMA >>> hint faults. >> >> We do need frequency information to determine whether a page is hot >> enough to be migrated to the fast memory (promotion). What PMU provided >> is just "recently" accessed pages, not "frequently" accessed pages. For >> current NUMA balancing implementation, please check >> NUMA_BALANCING_MEMORY_TIERING in should_numa_migrate_memory(). In >> general, it estimates the page access frequency via measuring the >> latency between page table scanning and page fault, the shorter the >> latency, the higher the frequency. This isn't perfect, but provides a >> starting point. You need to consider how to get frequency information >> via PMU. For example, you may count access number for each page, aging >> them periodically, and get hot threshold via some statistics. > > For the tiered memory hot page promotion case of NUMA balancing, we will > have to maintain frequency information in software when such information > isn't available from the hardware. Yes. It's challenging to calculate frequency information. Please consider how to do that. Best Regards, Huang, Ying
