On Fri, Jun 08, 2018 at 10:49:03AM +0200, Jirka Hladky wrote: > Hi Mel, > > automatic NUMA balancing doesn't run long enough to migrate all the > > memory. That would definitely be the case for STREAM. > > This could explain the behavior we observe. stream is running ~20 seconds > at the moment. I can easily change the runtime by changing the number of > iterations. What is the time period when you expect the memory to be fully > migrated? > Unknown and unknowable. It depends entirely on the reference pattern of the different threads. If they are fully parallelised with private buffers that are page-aligned then I expect it to be quick (to pass the 2-reference filter). If threads are sharing data on a 4K (base page case) or 2M boundary (THP enabled) then it may take longer as two or more threads will disagree on what the appropriate placement for a page is. > I have now checked numastat logs and after 15 seconds I see roughly 80MiB > out of 200MiB of the allocated memory migrated for each of 10 processes > which have changed the NUMA CPU node after started. This is on 2 > socket Gold 6126 CPU @ 2.60GHz server with DDR4 2666 MHz. That's 800 MiB of > memory migrated in 15 seconds which is results in the average migration > rate of 50MiB/s - is this an expected value? > I expect that to be far short of the capabilities of the machine. Again, migrations can be delayed indefinitely if threads have buffers that are not page-aligned (4K or 2M depending). -- Mel Gorman SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
