On Fri, Jul 12, 2019 at 06:48:05PM +0800, Huang, Ying wrote: > > Ordinarily I would hope that the patch was motivated by observed > > behaviour so you have a metric for goodness. However, for NUMA balancing > > I would typically run basic workloads first -- dbench, tbench, netperf, > > hackbench and pipetest. The objective would be to measure the degree > > automatic NUMA balancing is interfering with a basic workload to see if > > they patch reduces the number of minor faults incurred even though there > > is no NUMA balancing to be worried about. This measures the general > > overhead of a patch. If your reasoning is correct, you'd expect lower > > overhead. > > > > For balancing itself, I usually look at Andrea's original autonuma > > benchmark, NAS Parallel Benchmark (D class usually although C class for > > much older or smaller machines) and spec JBB 2005 and 2015. Of the JBB > > benchmarks, 2005 is usually more reasonable for evaluating NUMA balancing > > than 2015 is (which can be unstable for a variety of reasons). In this > > case, I would be looking at whether the overhead is reduced, whether the > > ratio of local hits is the same or improved and the primary metric of > > each (time to completion for Andrea's and NAS, throughput for JBB). > > > > Even if there is no change to locality and the primary metric but there > > is less scanning and overhead overall, it would still be an improvement. > > Thanks a lot for your detailed guidance. > No problem. > > If you have trouble doing such an evaluation, I'll queue tests if they > > are based on a patch that addresses the specific point of concern (scan > > period not updated) as it's still not obvious why flipping the logic of > > whether shared or private is considered was necessary. > > I can do the evaluation, but it will take quite some time for me to > setup and run all these benchmarks. So if these benchmarks have already > been setup in your environment, so that your extra effort is minimal, it > will be great if you can queue tests for the patch. Feel free to reject > me for any inconvenience. > They're not setup as such, but my testing infrastructure is heavily automated so it's easy to do and I think it's worth looking at. If you update your patch to target just the scan period aspects, I'll queue it up and get back to you. It usually takes a few days for the automation to finish whatever it's doing and pick up a patch for evaluation. -- Mel Gorman SUSE Labs
