On Tue, 13 Jun 2017 13:09:26 +0200 Daniel Borkmann <borkmann@xxxxxxxxxxxxx> wrote: > On 06/13/2017 12:30 PM, Jesper Dangaard Brouer wrote: > > Hi Fengguang and Daniel, > > > > I'm looking into adding some tests for tools/testing/selftests/bpf/. > > I'm a bit confused about the output format, as it seems to differ even > > among programs in the same directory (tools/testing/selftests/bpf/). > > > > I do see, that tools/testing/selftests/lib.mk have a standard output > > format via the program shell exit-code (copied Makefile syntax): > > > > exit_code && echo "selftests: $$BASENAME_TEST [PASS]" \ > > || echo "selftests: $$BASENAME_TEST [FAIL]" > > > > To Fengguang: > > How do your kbuild-robot hook into this? > > (1) Do you parse/grep after output? (egrep -e "^selftests: .* \[.*\]") > > (2) Or do you rely on the shell exit codes? > > > > A quick look at some of the bpf tests, show that they are not all using > > the exit codes to indicate failure (... which I guess is a bug we > > should fix?) > > Yep, if you have a chance please send a fix (otherwise I can also take > care of it). The way test_verifier handles it wrt exit code is correct, > and other tests should follow the same model. Thanks for spotting! Sure, I can fix it... I know you are a busy man :-) Looking at the code for test_verfier.c, I don't agree that it is using return codes correctly. The shell exit code should not be a negative number (you return -errors). /usr/include/stdlib.h #define EXIT_FAILURE 1 /* Failing exit status. */ #define EXIT_SUCCESS 0 /* Successful exit status. */ Further more the exit code is masked with 255, meaning that if you get 256 errors, and return -256 masked it will be 0, which would indicate success. > > p.s. I have noticed: Documentation/kselftest.txt, but it says nothing > > about output format used/required program exit codes. -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
