This series adds JSON input and output support to libnftables via libjansson. The first five patches prepare the existing code for the actual implementation which follows in patches 6 and 7. Patches 8 and 9 extend the simple Nftables Python class in py/nftables.py. The remaining ones deal with Python testsuite in tests/py: After a bit of cleanup, the last patch finally extends the testsuite to cover JSON, at least for positive rule tests. JSON output support is relatively trivial: If enabled, do_command_list() redirects to a new function do_command_list_json() which resides in src/json.c and mimicks the former's behaviour. For translating the elements of an nftables ruleset into JSON format, the various '_ops' structs are extended by a new callback which returns a libjansson json_t object. After the translation is finished, the resulting JSON object is dumped as a string to output_fp. To enable JSON output, libnftables API is extended by two new functions: * nft_ctx_output_get_json() and * nft_ctx_output_set_json() these toggle JSON output in the same manner as other libnftables configuration getter/setters do. JSON input support is implemented as an alternative parser to the existing lex/yacc-based one and resides in src/parser_json.c. It's two entry functions are: * nft_parse_json_buffer() and * nft_parse_json_filename(), which are called from the generic nft_run_cmd_from_*() functions libnftables API exports iff JSON output has been enabled. Note that this is rather a performance optimization: If JSON parsing fails early, i.e. the given buffer or file doesn't look like JSON input at all, regular syntax parsing is performed as fallback. So out of all combinations of JSON or standard input or output, only JSON input and standard output is not possible, but practical significance of that is probably negligible. Please note that this series of patches is not fully complete yet. The (known) missing pieces are: * JSON format documentation, * echo and monitor support, * a dedicated (but simple) test suite to increase test coverage. The latter two of those are already WiP, for the first one I'm a bit undecided in which form this should be done - dedicated man page, README, new wiki article? Heads-up for reviewers: In order to enforce constraints on which expression types are allowed in a certain place, parser_json makes use of context flags which are set by some routines (mainly json_parse_*_expr()) and then checked in json_parse_expr() (and elsewhere). This whole thing is so ugly it might burn your eyeballs and possibly everything behind them. I'm more than happy for any suggestion for improving this. You have been warned. Changes since v1: - Fix patches 6, 7 and patch 14 (former patch 13). - New patch 13. Changes since v2: - Rebased series onto current master branch. - Adjusted code to upstream changes in struct handle. - Minor improvement in JSON output debug code. - Aligned expected JSON output of one test in inet/ip_tcp.t with expected standard output. Phil Sutter (14): include/linux: Add required NFT_CT_MAX macro libnftables: Put bison parsing into dedicated functions libnftables: Make some arrays globally accessible libnftables: Make some functions globally accessible libnftables: Introduce a few helper functions libnftables: Implement JSON output support libnftables: Implement JSON parser py: Add getter/setter for echo output option py: Add JSON support to nftables Class tests/py: Reduce indenting level in nft-test.py tests/py: Simplify parsing of 'set' lines tests/py: Don't read expected payload for each table tests/py: Highlight offending parts in differences warnings tests/py: Support testing JSON input and output as well configure.ac | 14 +- include/ct.h | 4 + include/datatype.h | 5 + include/expression.h | 5 + include/fib.h | 3 + include/gmputil.h | 1 + include/json.h | 181 ++ include/linux/netfilter/nf_tables.h | 2 + include/meta.h | 4 + include/nftables.h | 3 + include/nftables/libnftables.h | 2 + include/rt.h | 2 + include/rule.h | 5 + include/statement.h | 10 + include/tcpopt.h | 4 + py/nftables.py | 70 +- src/Makefile.am | 5 + src/ct.c | 50 +- src/datatype.c | 11 +- src/expression.c | 17 +- src/exthdr.c | 2 + src/fib.c | 3 +- src/hash.c | 1 + src/json.c | 1564 ++++++++++++ src/libnftables.c | 124 +- src/main.c | 11 +- src/meta.c | 11 +- src/numgen.c | 1 + src/parser_json.c | 3141 +++++++++++++++++++++++++ src/payload.c | 3 + src/rt.c | 4 +- src/rule.c | 10 +- src/statement.c | 42 +- src/tcpopt.c | 2 +- tests/py/any/ct.t.json | 1352 +++++++++++ tests/py/any/ct.t.json.output | 575 +++++ tests/py/any/dup.t.json | 30 + tests/py/any/fwd.t.json | 32 + tests/py/any/fwd.t.json.output | 25 + tests/py/any/limit.t.json | 374 +++ tests/py/any/log.t.json | 169 ++ tests/py/any/log.t.json.output | 16 + tests/py/any/meta.t.json | 2418 +++++++++++++++++++ tests/py/any/meta.t.json.output | 577 +++++ tests/py/any/queue.t.json | 86 + tests/py/any/queue.t.json.output | 9 + tests/py/any/quota.t.json | 136 ++ tests/py/any/rawpayload.t.json | 157 ++ tests/py/any/rawpayload.t.json.output | 104 + tests/py/any/rt.t.json | 14 + tests/py/arp/arp.t.json | 877 +++++++ tests/py/arp/arp.t.json.output | 148 ++ tests/py/bridge/ether.t.json | 180 ++ tests/py/bridge/ether.t.json.output | 77 + tests/py/bridge/icmpX.t.json | 82 + tests/py/bridge/icmpX.t.json.output | 52 + tests/py/bridge/meta.t.json | 23 + tests/py/bridge/reject.t.json | 221 ++ tests/py/bridge/reject.t.json.output | 260 ++ tests/py/bridge/vlan.t.json | 468 ++++ tests/py/inet/ah.t.json | 557 +++++ tests/py/inet/comp.t.json | 316 +++ tests/py/inet/comp.t.json.output | 166 ++ tests/py/inet/ct.t.json | 39 + tests/py/inet/ct.t.json.output | 16 + tests/py/inet/dccp.t.json | 340 +++ tests/py/inet/dccp.t.json.output | 17 + tests/py/inet/esp.t.json | 256 ++ tests/py/inet/ether-ip.t.json | 85 + tests/py/inet/ether-ip.t.json.output | 40 + tests/py/inet/ether.t.json | 84 + tests/py/inet/ether.t.json.output | 29 + tests/py/inet/fib.t.json | 128 + tests/py/inet/fib.t.json.output | 39 + tests/py/inet/icmpX.t.json | 116 + tests/py/inet/icmpX.t.json.output | 86 + tests/py/inet/ip.t.json | 41 + tests/py/inet/ip_tcp.t.json | 158 ++ tests/py/inet/ip_tcp.t.json.output | 121 + tests/py/inet/map.t.json | 66 + tests/py/inet/map.t.json.output | 66 + tests/py/inet/meta.t.json | 198 ++ tests/py/inet/meta.t.json.output | 35 + tests/py/inet/reject.t.json | 240 ++ tests/py/inet/reject.t.json.output | 224 ++ tests/py/inet/rt.t.json | 56 + tests/py/inet/rt.t.json.output | 23 + tests/py/inet/sctp.t.json | 594 +++++ tests/py/inet/tcp.t.json | 1552 ++++++++++++ tests/py/inet/tcp.t.json.output | 134 ++ tests/py/inet/tcpopt.t.json | 418 ++++ tests/py/inet/tcpopt.t.json.output | 30 + tests/py/inet/udp.t.json | 730 ++++++ tests/py/inet/udplite.t.json | 526 +++++ tests/py/ip/ct.t.json | 213 ++ tests/py/ip/ct.t.json.output | 25 + tests/py/ip/dnat.t.json | 255 ++ tests/py/ip/dnat.t.json.output | 65 + tests/py/ip/dup.t.json | 46 + tests/py/ip/ether.t.json | 151 ++ tests/py/ip/ether.t.json.output | 40 + tests/py/ip/flowtable.t.json | 23 + tests/py/ip/hash.t.json | 230 ++ tests/py/ip/icmp.t.json | 1382 +++++++++++ tests/py/ip/icmp.t.json.output | 85 + tests/py/ip/ip.t.json | 1781 ++++++++++++++ tests/py/ip/ip.t.json.output | 226 ++ tests/py/ip/ip_tcp.t.json | 60 + tests/py/ip/ip_tcp.t.json.output | 49 + tests/py/ip/masquerade.t.json | 411 ++++ tests/py/ip/masquerade.t.json.output | 118 + tests/py/ip/meta.t.json | 99 + tests/py/ip/meta.t.json.output | 45 + tests/py/ip/numgen.t.json | 99 + tests/py/ip/numgen.t.json.output | 82 + tests/py/ip/objects.t.json | 185 ++ tests/py/ip/objects.t.json.output | 64 + tests/py/ip/redirect.t.json | 610 +++++ tests/py/ip/redirect.t.json.output | 140 ++ tests/py/ip/reject.t.json | 94 + tests/py/ip/reject.t.json.output | 24 + tests/py/ip/rt.t.json | 15 + tests/py/ip/sets.t.json | 185 ++ tests/py/ip/snat.t.json | 161 ++ tests/py/ip/snat.t.json.output | 69 + tests/py/ip/tcp.t.json | 59 + tests/py/ip/tcp.t.json.output | 48 + tests/py/ip/tcpopt.t.json | 390 +++ tests/py/ip/tcpopt.t.json.output | 15 + tests/py/ip6/dnat.t.json | 75 + tests/py/ip6/dst.t.json | 386 +++ tests/py/ip6/dst.t.json.output | 86 + tests/py/ip6/dup.t.json | 46 + tests/py/ip6/ether.t.json | 151 ++ tests/py/ip6/ether.t.json.output | 40 + tests/py/ip6/exthdr.t.json | 172 ++ tests/py/ip6/exthdr.t.json.output | 56 + tests/py/ip6/flowtable.t.json | 60 + tests/py/ip6/frag.t.json | 624 +++++ tests/py/ip6/frag.t.json.output | 114 + tests/py/ip6/hbh.t.json | 386 +++ tests/py/ip6/hbh.t.json.output | 66 + tests/py/ip6/icmpv6.t.json | 1257 ++++++++++ tests/py/ip6/icmpv6.t.json.output | 92 + tests/py/ip6/ip6.t.json | 1613 +++++++++++++ tests/py/ip6/ip6.t.json.output | 361 +++ tests/py/ip6/map.t.json | 38 + tests/py/ip6/map.t.json.output | 38 + tests/py/ip6/masquerade.t.json | 405 ++++ tests/py/ip6/masquerade.t.json.output | 94 + tests/py/ip6/meta.t.json | 99 + tests/py/ip6/meta.t.json.output | 45 + tests/py/ip6/mh.t.json | 781 ++++++ tests/py/ip6/mh.t.json.output | 86 + tests/py/ip6/redirect.t.json | 576 +++++ tests/py/ip6/redirect.t.json.output | 140 ++ tests/py/ip6/reject.t.json | 84 + tests/py/ip6/reject.t.json.output | 24 + tests/py/ip6/rt.t.json | 717 ++++++ tests/py/ip6/rt.t.json.output | 86 + tests/py/ip6/rt0.t.json | 15 + tests/py/ip6/sets.t.json | 91 + tests/py/ip6/snat.t.json | 52 + tests/py/ip6/srh.t.json | 183 ++ tests/py/ip6/srh.t.json.output | 21 + tests/py/ip6/tcpopt.t.json | 390 +++ tests/py/ip6/tcpopt.t.json.output | 15 + tests/py/ip6/vmap.t.json | 1037 ++++++++ tests/py/ip6/vmap.t.json.output | 336 +++ tests/py/nft-test.py | 381 ++- 170 files changed, 40010 insertions(+), 158 deletions(-) create mode 100644 include/json.h create mode 100644 src/json.c create mode 100644 src/parser_json.c create mode 100644 tests/py/any/ct.t.json create mode 100644 tests/py/any/ct.t.json.output create mode 100644 tests/py/any/dup.t.json create mode 100644 tests/py/any/fwd.t.json create mode 100644 tests/py/any/fwd.t.json.output create mode 100644 tests/py/any/limit.t.json create mode 100644 tests/py/any/log.t.json create mode 100644 tests/py/any/log.t.json.output create mode 100644 tests/py/any/meta.t.json create mode 100644 tests/py/any/meta.t.json.output create mode 100644 tests/py/any/queue.t.json create mode 100644 tests/py/any/queue.t.json.output create mode 100644 tests/py/any/quota.t.json create mode 100644 tests/py/any/rawpayload.t.json create mode 100644 tests/py/any/rawpayload.t.json.output create mode 100644 tests/py/any/rt.t.json create mode 100644 tests/py/arp/arp.t.json create mode 100644 tests/py/arp/arp.t.json.output create mode 100644 tests/py/bridge/ether.t.json create mode 100644 tests/py/bridge/ether.t.json.output create mode 100644 tests/py/bridge/icmpX.t.json create mode 100644 tests/py/bridge/icmpX.t.json.output create mode 100644 tests/py/bridge/meta.t.json create mode 100644 tests/py/bridge/reject.t.json create mode 100644 tests/py/bridge/reject.t.json.output create mode 100644 tests/py/bridge/vlan.t.json create mode 100644 tests/py/inet/ah.t.json create mode 100644 tests/py/inet/comp.t.json create mode 100644 tests/py/inet/comp.t.json.output create mode 100644 tests/py/inet/ct.t.json create mode 100644 tests/py/inet/ct.t.json.output create mode 100644 tests/py/inet/dccp.t.json create mode 100644 tests/py/inet/dccp.t.json.output create mode 100644 tests/py/inet/esp.t.json create mode 100644 tests/py/inet/ether-ip.t.json create mode 100644 tests/py/inet/ether-ip.t.json.output create mode 100644 tests/py/inet/ether.t.json create mode 100644 tests/py/inet/ether.t.json.output create mode 100644 tests/py/inet/fib.t.json create mode 100644 tests/py/inet/fib.t.json.output create mode 100644 tests/py/inet/icmpX.t.json create mode 100644 tests/py/inet/icmpX.t.json.output create mode 100644 tests/py/inet/ip.t.json create mode 100644 tests/py/inet/ip_tcp.t.json create mode 100644 tests/py/inet/ip_tcp.t.json.output create mode 100644 tests/py/inet/map.t.json create mode 100644 tests/py/inet/map.t.json.output create mode 100644 tests/py/inet/meta.t.json create mode 100644 tests/py/inet/meta.t.json.output create mode 100644 tests/py/inet/reject.t.json create mode 100644 tests/py/inet/reject.t.json.output create mode 100644 tests/py/inet/rt.t.json create mode 100644 tests/py/inet/rt.t.json.output create mode 100644 tests/py/inet/sctp.t.json create mode 100644 tests/py/inet/tcp.t.json create mode 100644 tests/py/inet/tcp.t.json.output create mode 100644 tests/py/inet/tcpopt.t.json create mode 100644 tests/py/inet/tcpopt.t.json.output create mode 100644 tests/py/inet/udp.t.json create mode 100644 tests/py/inet/udplite.t.json create mode 100644 tests/py/ip/ct.t.json create mode 100644 tests/py/ip/ct.t.json.output create mode 100644 tests/py/ip/dnat.t.json create mode 100644 tests/py/ip/dnat.t.json.output create mode 100644 tests/py/ip/dup.t.json create mode 100644 tests/py/ip/ether.t.json create mode 100644 tests/py/ip/ether.t.json.output create mode 100644 tests/py/ip/flowtable.t.json create mode 100644 tests/py/ip/hash.t.json create mode 100644 tests/py/ip/icmp.t.json create mode 100644 tests/py/ip/icmp.t.json.output create mode 100644 tests/py/ip/ip.t.json create mode 100644 tests/py/ip/ip.t.json.output create mode 100644 tests/py/ip/ip_tcp.t.json create mode 100644 tests/py/ip/ip_tcp.t.json.output create mode 100644 tests/py/ip/masquerade.t.json create mode 100644 tests/py/ip/masquerade.t.json.output create mode 100644 tests/py/ip/meta.t.json create mode 100644 tests/py/ip/meta.t.json.output create mode 100644 tests/py/ip/numgen.t.json create mode 100644 tests/py/ip/numgen.t.json.output create mode 100644 tests/py/ip/objects.t.json create mode 100644 tests/py/ip/objects.t.json.output create mode 100644 tests/py/ip/redirect.t.json create mode 100644 tests/py/ip/redirect.t.json.output create mode 100644 tests/py/ip/reject.t.json create mode 100644 tests/py/ip/reject.t.json.output create mode 100644 tests/py/ip/rt.t.json create mode 100644 tests/py/ip/sets.t.json create mode 100644 tests/py/ip/snat.t.json create mode 100644 tests/py/ip/snat.t.json.output create mode 100644 tests/py/ip/tcp.t.json create mode 100644 tests/py/ip/tcp.t.json.output create mode 100644 tests/py/ip/tcpopt.t.json create mode 100644 tests/py/ip/tcpopt.t.json.output create mode 100644 tests/py/ip6/dnat.t.json create mode 100644 tests/py/ip6/dst.t.json create mode 100644 tests/py/ip6/dst.t.json.output create mode 100644 tests/py/ip6/dup.t.json create mode 100644 tests/py/ip6/ether.t.json create mode 100644 tests/py/ip6/ether.t.json.output create mode 100644 tests/py/ip6/exthdr.t.json create mode 100644 tests/py/ip6/exthdr.t.json.output create mode 100644 tests/py/ip6/flowtable.t.json create mode 100644 tests/py/ip6/frag.t.json create mode 100644 tests/py/ip6/frag.t.json.output create mode 100644 tests/py/ip6/hbh.t.json create mode 100644 tests/py/ip6/hbh.t.json.output create mode 100644 tests/py/ip6/icmpv6.t.json create mode 100644 tests/py/ip6/icmpv6.t.json.output create mode 100644 tests/py/ip6/ip6.t.json create mode 100644 tests/py/ip6/ip6.t.json.output create mode 100644 tests/py/ip6/map.t.json create mode 100644 tests/py/ip6/map.t.json.output create mode 100644 tests/py/ip6/masquerade.t.json create mode 100644 tests/py/ip6/masquerade.t.json.output create mode 100644 tests/py/ip6/meta.t.json create mode 100644 tests/py/ip6/meta.t.json.output create mode 100644 tests/py/ip6/mh.t.json create mode 100644 tests/py/ip6/mh.t.json.output create mode 100644 tests/py/ip6/redirect.t.json create mode 100644 tests/py/ip6/redirect.t.json.output create mode 100644 tests/py/ip6/reject.t.json create mode 100644 tests/py/ip6/reject.t.json.output create mode 100644 tests/py/ip6/rt.t.json create mode 100644 tests/py/ip6/rt.t.json.output create mode 100644 tests/py/ip6/rt0.t.json create mode 100644 tests/py/ip6/sets.t.json create mode 100644 tests/py/ip6/snat.t.json create mode 100644 tests/py/ip6/srh.t.json create mode 100644 tests/py/ip6/srh.t.json.output create mode 100644 tests/py/ip6/tcpopt.t.json create mode 100644 tests/py/ip6/tcpopt.t.json.output create mode 100644 tests/py/ip6/vmap.t.json create mode 100644 tests/py/ip6/vmap.t.json.output -- 2.17.0 -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
