On 04/07/2014 07:47 PM, Christoph Lameter wrote:
On Mon, 7 Apr 2014, Andi Kleen wrote:
* starts a binary with the specified numa memory policy using
numactl (or a like):
numactl --interleave=all get_some_memory_with_malloc_and_write_it
* `sleep` for few seconds
* numastat > /tmp/after
* compares /tmp/before and /tmp/after to check that the numa policy
was applied the right way
But the problem is that on a host with many NUMA nodes (8) the process
of updating that numastats statistics takes some time. Even 10 seconds
may be not enough. Therefore the test fails.
Is there a direct or indirect way to force the kernel to update the
NUMA statistics?
Not currently. It depends on how much memory you have and subsequent
operations. I guess would need to add one.
The kernel vm statistics are brought up to date with the default
settings every 2 seconds.
The interval is controlled via /proc/sys/vm/stat_interval
Check the value that you have setup there.
Thank you, Andi, Christoph.
In my setup stat_interval is 1.
Please, look at this reproducer:
#!/bin/bash
sum_pages()
{
local i
ret=0
for i in $@; do
ret=$(( $ret + $i ))
done
}
ret=0
for i in `seq 20`; do
sum_pages $( numastat | grep interleav | cut -d ' ' -f 2-)
val_before=$ret
numactl --interleave=all support_numa 2
sleep 2
sum_pages $( numastat | grep interleav | cut -d ' ' -f 2-)
val_after=$ret
echo "$i: $(( $val_after - $val_before))"
done
On a two-node system it prints:
1: 294
2: 294
3: 295
4: 294
5: 294
6: 295
7: 294
8: 294
9: 293
10: 293
11: 294
12: 295
13: 296
14: 293
15: 295
16: 294
17: 295
18: 294
19: 294
20: 294
i.e. everything is ok.
But on an eight-node system:
1: 173
2: 0
3: 0
4: 173
5: 173
6: 0
7: 173
8: 173
9: 0
10: 0
11: 173
12: 0
13: 173
14: 0
15: 346
16: 0
17: 0
18: 89
19: 0
20: 173
So in general we can't rely on stat_interval value. Correct?
--
To unsubscribe from this list: send the line "unsubscribe linux-numa" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
