> On 1/27/22 7:38 AM, Jose E. Marchesi wrote:
>>
>>> On 12/20/21 1:49 AM, Jose E. Marchesi wrote:
>>>>
>>>>> On 12/17/21 5:44 PM, Alexei Starovoitov wrote:
>>>>>> On Fri, Dec 17, 2021 at 11:40:10AM +0100, Jose E. Marchesi wrote:
>>>>>>>
>>>>>>> 2) The need for DWARF to convey free-text tags on certain elements, such
>>>>>>> as members of struct types.
>>>>>>>
>>>>>>> The motivation for this was originally the way the Linux kernel
>>>>>>> generates its BTF information, using pahole, using DWARF as a source.
>>>>>>> As we discussed in our last exchange on this topic, this is
>>>>>>> accidental, i.e. if the kernel switched to generate BTF directly from
>>>>>>> the compiler and the linker could merge/deduplicate BTF, there would
>>>>>>> be no need for using DWARF to act as the "unwilling conveyer" of this
>>>>>>> information. There are additional benefits of this second approach.
>>>>>>> Thats why we didn't plan to add these extended DWARF DIEs to GCC.
>>>>>>>
>>>>>>> However, it now seems that a DWARF consumer, the drgn project, would
>>>>>>> also benefit from having such a support in DWARF to distinguish
>>>>>>> between different kind of pointers.
>>>>>> drgn can use .percpu section in vmlinux for global percpu vars.
>>>>>> For pointers the annotation is indeed necessary.
>>>>>>
>>>>>>> So it seems to me that now we have two use-cases for adding support
>>>>>>> for these free-text tags to DWARF, as a proper extension to the
>>>>>>> format, strictly unrelated to BTF, BPF or even the kernel, since:
>>>>>>> - This is not kernel specific.
>>>>>>> - This is not directly related to BTF.
>>>>>>> - This is not directly related to BPF.
>>>>>> __percpu annotation is kernel specific.
>>>>>> __user and __rcu are kernel specific too.
>>>>>> Only BPF and BTF can meaningfully consume all three.
>>>>>> drgn can consume __percpu.
>>>>>> In that sense if GCC follows LLVM and emits compiler specific DWARF
>>>>>> tag
>>>>>> pahole can convert it to the same BTF regardless whether kernel
>>>>>> was compiled with clang or gcc.
>>>>>> drgn can consume dwarf generated by clang or gcc as well even when BTF
>>>>>> is not there. That is the fastest way forward.
>>>>>> In that sense it would be nice to have common DWARF tag for pointer
>>>>>> annotations, but it's not mandatory. The time is the most valuable asset.
>>>>>> Implementing GCC specific DWARF tag doesn't require committee voting
>>>>>> and the mailing list bikeshedding.
>>>>>>
>>>>>>> 3) Addition of C-family language-level constructions to specify
>>>>>>> free-text tags on certain language elements, such as struct fields.
>>>>>>>
>>>>>>> These are the attributes, or built-ins or whatever syntax.
>>>>>>>
>>>>>>> Note that, strictly speaking:
>>>>>>> - This is orthogonal to both DWARF and BTF, and any other supported
>>>>>>> debugging format, which may or may not be expressive enough to
>>>>>>> convey the free-form text tag.
>>>>>>> - This is not specific to BPF.
>>>>>>>
>>>>>>> Therefore I would avoid any reference to BTF or BPF in the attribute
>>>>>>> names. Something like `__attribute__((btf_tag("arbitrary_str")))'
>>>>>>> makes very little sense to me; the attribute name ought to be more
>>>>>>> generic.
>>>>>> Let's agree to disagree.
>>>>>> When BPF ISA was designed we didn't go to Intel, Arm, Mips, etc in order to
>>>>>> come up with the best ISA that would JIT to those architectures the best
>>>>>> possible way. Same thing with btf_tag. Today it is specific to BTF and BPF
>>>>>> only. Hence it's called this way. Whenever actual users will appear that need
>>>>>> free-text tags on a struct field then and only then will be the time to discuss
>>>>>> generic tag name. Just because "free-text tag on a struct field" sounds generic
>>>>>> it doesn't mean that it has any use case beyond what we're using it for in BPF
>>>>>> land. It goes back to the point of coding now instead of talking about coding.
>>>>>> If gcc wants to call it __attribute__((my_precious_gcc_tag("arbitrary_str")))
>>>>>> go ahead and code it this way. The include/linux/compiler.h can accommodate it.
>>>>>
>>>>> Just want to add a little bit context for this. In the beginning when
>>>>> we proposed to add the attribute, we named as a generic name like
>>>>> 'tag' (or something like that). But eventually upstream suggested
>>>>> 'btf_tag' since the use case we proposed is for bpf. At that point, we
>>>>> don't know drgn use cases yet. Even with that, the use cases are still
>>>>> just for linux kernel.
>>>>>
>>>>> At that time, some *similar* use cases did came up, e.g., for
>>>>> swift<->C++ conversion encoding ("tag name", "attribute info") for
>>>>> attributes in the source code, will help a lot. But they will use a
>>>>> different "tag name" than btf_tag to differentiate.
>>>> Thanks for the info.
>>>> I find it very interesting that the LLVM people prefers to have
>>>> several
>>>> "use case specific" tag names instead of something more generic, which
>>>> is the exact opposite of what I would have done :) They may have
>>>> appealing reasons for doing so. Do you have a pointer to the dicussion
>>>> you had upstream at hand?
>>>> Anyway, I will taste the waters with the other GCC hackers about
>>>> both
>>>> DIEs and attribute and see what we can come out with. Thanks again for
>>>> reaching out Yonghong.
>>>
>>> Hi, Jose,
>>>
>>> Any progress on gcc btf_tag support discussion? If possible, could
>>> you add me to the discussion mailing list so I may help to move
>>> the project forward? Thanks a lot!
>> We are in the process of implementing the support of the BTF
>> extensions
>> (which is done) and the C language attributes (which is WIP.)
>
> Sounds good. I am happy to answer questions if you have any.
>
>> I haven't started the discussion about DWARF yet. Will do shortly.
>> You
>> will be in CC :)
>
> Thanks a lot, Jose! I am looking forward to the discussion.
Just a heads-up.
We are still working on the GCC implementation of the tags. Having some
difficulties with the ordering of the C type attributes.
Regarding the DWARF part, GCC uses DWARF as the internal "canonical"
debug info, and the BTF is generated from it. This means we had to add
a DWARF DIE for the pointer tag qualifier anyway in order to convey the
info to BTF. So now it is just a matter of emitting it along with the
rest of the DWARF.
