On Mon, 2023-11-06 at 15:04 +0100, Pablo Neira Ayuso wrote: > On Mon, Nov 06, 2023 at 02:56:18PM +0100, Thomas Haller wrote: > > On Fri, 2023-11-03 at 19:26 +0100, Thomas Haller wrote: > > > Generate and add ".json-nft" files. These files contain the > > > output of > > > `nft -j list ruleset` after the test. Also, "test-wrapper.sh" > > > will > > > compare the current ruleset against the ".json-nft" files and > > > test > > > them > > > with "nft -j --check -f $FILE`. These are useful extra tests, > > > that we > > > almost get for free. > > > > > > Note that for some JSON dumps, `nft -f --check` fails (or prints > > > something). For those tests no *.json-nft file is added. The bugs > > > needs > > > to be fixed first. > > > > > > An example of such an issue is: > > > > > > $ DUMPGEN=all ./tests/shell/run-tests.sh > > > tests/shell/testcases/maps/nat_addr_port > > > > > > which gives a file "rc-failed-chkdump" with > > > > > > Command `./tests/shell/../../src/nft -j --check -f > > > "tests/shell/testcases/maps/dumps/nat_addr_port.json-nft"` failed > > > >>>> > > > internal:0:0-0: Error: Invalid map type 'ipv4_addr . > > > inet_service'. > > > > > > internal:0:0-0: Error: Parsing command array at index 3 > > > failed. > > > > > > internal:0:0-0: Error: unqualified type integer specified in > > > map > > > definition. Try "typeof expression" instead of "type datatype". > > > > > > <<<< > > > > > > Tests like "tests/shell/testcases/nft-f/0012different_defines_0" > > > and > > > "tests/shell/testcases/nft-f/0024priority_0" also don't get a > > > .json- > > > nft > > > dump yet, because their output is not stable. That needs fixing > > > too. > > > > > > Cc: Pablo Neira Ayuso <pablo@xxxxxxxxxxxxx> > > > Cc: Florian Westphal <fw@xxxxxxxxx> > > > Signed-off-by: Thomas Haller <thaller@xxxxxxxxxx> > > > --- > > ... > > > tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft > > > | > > ... > > > tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft > > > create mode 100644 > > > > > > "tests/shell/testcases/sets/dumps/0062set_connlimit_0.json-nft" > > need to > > be dropped from this patch. > > > > Otherwise, > > > > make && ./tests/shell/run-tests.sh > > tests/shell/testcases/sets/0062set_connlimit_0 -V > > > > fails (in valgrind mode). > > I have to fix 0062set_connlimit_0, the listing fails because the GC > is > fast enough to remove the entry that just got added, because it has > no > conntrack entries. > > Maybe valgrind is just getting things slowier there to trigger what > I can already reproduce here on a VM? that sounds very plausible. By adding a `sleep 1` at the end of "tests/shell/testcases/sets/0062set_connlimit_0", the dump "tests/shell/testcases/sets/dumps/0062set_connlimit_0.nft" also no longer matches. Thomas
