On 11/11/22 1:55 PM, Eduard Zingerman wrote:
On Fri, 2022-10-28 at 11:56 -0700, Yonghong Song wrote:
[...]
Ok, could we change the problem to detecting if some type is defined.
Would it be possible to have something like
#if !__is_type_defined(struct abc)
struct abc {
};
#endif
I think we talked about this and there were problems with this
approach, but I don't remember details and how insurmountable the
problem is. Having a way to check whether some type is defined would
be very useful even outside of -target bpf parlance, though, so maybe
it's the problem worth attacking?
Yes, we discussed this before. This will need to add additional work
in preprocessor. I just made a discussion topic in llvm discourse
https://discourse.llvm.org/t/add-a-type-checking-macro-is-type-defined-type/66268
Let us see whether we can get some upstream agreement or not.
I did a small investigation of this feature.
The main pre-requirement is construction of the symbol table during
source code pre-processing, which implies necessity to parse the
source code at the same time. It is technically possible in clang, as
lexing, pre-processing and AST construction happens at the same time
when in compilation mode.
The prototype is available here [1], it includes:
- Change in the pre-processor that adds an optional callback
"IsTypeDefinedFn" & necessary parsing of __is_type_defined
construct.
- Change in Sema module (responsible for parsing/AST & symbol table)
that installs the appropriate "IsTypeDefinedFn" in the pre-processor
instance.
However, this prototype builds a backward dependency between
pre-processor and semantic analysis. There are currently no such
dependencies in the clang code base.
This makes it impossible to do pre-processing and compilation
separately, e.g. consider the following example:
$ cat test.c
struct foo { int x; };
#if __is_type_defined(foo)
const int x = 1;
#else
const int x = 2;
#endif
$ clang -cc1 -ast-print test.c -o -
struct foo {
int x;
};
const int x = 1;
$ clang -E test.c -o -
# ... some line directives ...
struct foo { int x; };
const int x = 2;
Is it any chance '-E' could output the same one as '-cc1 -ast-print'?
That is, even with -E we could do some semantics analysis
as well, using either current clang semantics analysis or creating
an minimal version of sema analysis in preprocessor itself?
Note that __is_type_defined is computed to different value in the
first and second calls. This is so because semantic analysis (AST,
symbol table) is not done for -E.
It also breaks that C11 standard which clearly separates
pre-processing and semantic analysis phases, see [2] 5.1.1.2.
So, my conclusion is as follows: this is technically possible in clang
but has no chance to reach llvm upstream.
Thanks,
Eduard
[1] https://github.com/llvm/llvm-project/compare/main...eddyz87:llvm-project:is-type-defined-experiment
[2] https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1548.pdf
BTW, I suggest splitting libbpf btf_dedup and btf_dump changes into a
separate series and sending them as non-RFC sooner. Those improvements
are independent of all the header guards stuff, let's get them landed
sooner.
After some discussion with Alexei and Yonghong I'd like to request
your comments regarding a somewhat brittle and partial solution to
this issue that relies on adding `#ifndef FOO_H ... #endif` guards in
the generated `vmlinux.h`.
[...]
Eduard Zingerman (12):
libbpf: Deduplicate unambigous standalone forward declarations
selftests/bpf: Tests for standalone forward BTF declarations
deduplication
libbpf: Support for BTF_DECL_TAG dump in C format
selftests/bpf: Tests for BTF_DECL_TAG dump in C format
libbpf: Header guards for selected data structures in vmlinux.h
selftests/bpf: Tests for header guards printing in BTF dump
bpftool: Enable header guards generation
kbuild: Script to infer header guard values for uapi headers
kbuild: Header guards for types from include/uapi/*.h in kernel BTF
selftests/bpf: Script to verify uapi headers usage with vmlinux.h
selftests/bpf: Known good uapi headers for test_uapi_headers.py
selftests/bpf: script for infer_header_guards.pl testing
scripts/infer_header_guards.pl | 191 +++++
scripts/link-vmlinux.sh | 13 +-
tools/bpf/bpftool/btf.c | 4 +-
tools/lib/bpf/btf.c | 178 ++++-
tools/lib/bpf/btf.h | 7 +-
tools/lib/bpf/btf_dump.c | 232 +++++-
.../selftests/bpf/good_uapi_headers.txt | 677 ++++++++++++++++++
tools/testing/selftests/bpf/prog_tests/btf.c | 152 ++++
.../selftests/bpf/prog_tests/btf_dump.c | 11 +-
.../bpf/progs/btf_dump_test_case_decl_tag.c | 39 +
.../progs/btf_dump_test_case_header_guards.c | 94 +++
.../bpf/test_uapi_header_guards_infer.sh | 33 +
.../selftests/bpf/test_uapi_headers.py | 197 +++++
13 files changed, 1816 insertions(+), 12 deletions(-)
create mode 100755 scripts/infer_header_guards.pl
create mode 100644 tools/testing/selftests/bpf/good_uapi_headers.txt
create mode 100644 tools/testing/selftests/bpf/progs/btf_dump_test_case_decl_tag.c
create mode 100644 tools/testing/selftests/bpf/progs/btf_dump_test_case_header_guards.c
create mode 100755 tools/testing/selftests/bpf/test_uapi_header_guards_infer.sh
create mode 100755 tools/testing/selftests/bpf/test_uapi_headers.py
--
2.34.1
