On Fri, Feb 09, 2018 at 03:01:57PM +0100, Daniel Borkmann wrote:
> On 02/09/2018 06:14 AM, Li Zhijian wrote:
> > Hi
> >
> > INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> > when it fails, the details are like
>
> Sorry for the late reply and thanks for reporting! More below:
>
> > ------------------
> > 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> > 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> > 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> > 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> > 906 Aborted
> > 907 not ok 1..3 selftests: test_maps [FAIL]
> > ------------------
> >
> > After a simply looking at the code, looks it's related to the cpu number and system memory.
> >
> > below are the result under different platform
> > 1. Good
> > model: Sandy Bridge
> > nr_node: 1
> > nr_cpu: 4
> > memory: 6G
> >
> > 2. Good
> > model: qemu-system-x86_64 -enable-kvm
> > nr_cpu: 2
> > memory: 4G
> >
> > 3. Bad
> > model: Ivytown Ivy Bridge-EP
> > nr_cpu: 48
> > memory: 64G
> >
> > 4. Bad
> > model: Skylake
> > nr_cpu: 104
> > memory: 64G
> >
> > I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> > ------------
> > lizhijian@haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> > diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> > index 040356e..b788ca1 100644
> > --- a/tools/testing/selftests/bpf/test_maps.c
> > +++ b/tools/testing/selftests/bpf/test_maps.c
> > @@ -960,7 +960,7 @@ static void test_map_stress(void)
> > {
> > run_parallel(100, test_hashmap, NULL);
> > run_parallel(100, test_hashmap_percpu, NULL);
> > - run_parallel(100, test_hashmap_sizes, NULL);
> > + run_parallel(10, test_hashmap_sizes, NULL);
> > run_parallel(100, test_hashmap_walk, NULL);
> >
> > run_parallel(100, test_arraymap, NULL);
>
> Unless Alexei has some better idea, I think if the bpf_create_map() error in
> the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
> all other cases we should however. So probably makes sense to just check for
> errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
> to keep trying under stress. Feel free to send a patch, Li.
that's probably good path for now.
I also see that test_maps fails on freshly booted kernel with such assert,
but then restarting test_maps again works and repeated runs succeed too.
I suspect there is a deeper issue here related to memory allocation.
Either slab or percpu allocator are behaving funky.
It needs to be further debugged.
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
