Bharata B Rao <bharata@xxxxxxx> writes: > On 03-Apr-24 2:10 PM, Huang, Ying wrote: >> Bharata B Rao <bharata@xxxxxxx> writes: >> >>> On 02-Apr-24 7:33 AM, Huang, Ying wrote: >>>> Bharata B Rao <bharata@xxxxxxx> writes: >>>> >>>>> On 29-Mar-24 6:44 AM, Huang, Ying wrote: >>>>>> Bharata B Rao <bharata@xxxxxxx> writes: >>>>> <snip> >>>>>>> I don't think the pages are cold but rather the existing mechanism fails >>>>>>> to categorize them as hot. This is because the pages were scanned way >>>>>>> before the accesses start happening. When repeated accesses are made to >>>>>>> a chunk of memory that has been scanned a while back, none of those >>>>>>> accesses get classified as hot because the scan time is way behind >>>>>>> the current access time. That's the reason we are seeing the value >>>>>>> of latency ranging from 20s to 630s as shown above. >>>>>> >>>>>> If repeated accesses continue, the page will be identified as hot when >>>>>> it is scanned next time even if we don't expand the threshold range. If >>>>>> the repeated accesses only last very short time, it makes little sense >>>>>> to identify the pages as hot. Right? >>>>> >>>>> The total allocated memory here is 192G and the chunk size is 1G. Each >>>>> time one such 1G chunk is taken up randomly for generating memory accesses. >>>>> Within that 1G, 262144 random accesses are performed and 262144 such >>>>> accesses are repeated for 512 times. I thought that should be enough >>>>> to classify that chunk of memory as hot. >>>> >>>> IIUC, some pages are accessed in very short time (maybe within 1ms). >>>> This isn't repeated access in a long period. I think that pages >>>> accessed repeatedly in a long period are good candidates for promoting. >>>> But pages accessed frequently in only very short time aren't. >>> >>> Here are the numbers for the 192nd chunk: >>> >>> Each iteration of 262144 random accesses takes around ~10ms >>> 512 such iterations are taking ~5s >>> numa_scan_seq is 16 when this chunk is accessed. >>> And no page promotions were done from this chunk. All the >>> time should_numa_migrate_memory() found the NUMA hint fault >>> latency to be higher than threshold. >>> >>> Are these time periods considered too short for the pages >>> to be detected as hot and promoted? >> >> Yes. I think so. This is burst accessing, not repeated accessing. >> IIUC, NUMA balancing based promotion only works for repeated accessing >> for long time, for example, >100s. > > Hmm... When a page is accessed 512 times over a period of 5s and it is > still not detected as hot. This is understandable if fresh scanning couldn't > be done as the accesses were bursty and hence they couldn't be captured via > NUMA hint faults. But here the access captured via hint fault is being rejected > as not hot because the scanning was done a while back. But I do see the challenge > here since we depend on scanning time to obtain the frequency-of-access metric. Consider some pages that will be accessed once every 1 hour, should we consider it hot or not? Will your proposed method deal with that correctly? > BTW, for the above same scenario with numa_balancing_mode=1, the remote > accesses get detected and migration to source node is tried. It is a different > matter that eventually pages can't be migrated in this specific scenario as > the src node is already full. -- Best Regards, Huang, Ying
