Hi Kumar, On Tue, Aug 23, 2022 at 04:29:17AM +0200, Kumar Kartikeya Dwivedi wrote: > On Tue, 23 Aug 2022 at 04:19, Daniel Xu <dxu@xxxxxxxxx> wrote: [...] > > There's also some other issues I'm uncovering with duplicate BTF IDs for > > nf_conn. Might have to do a lookup by name instead of the BTF_ID_LIST(). > > > > I think I also hit this problem back when I was working on these > patches, is it similar to this? > https://lore.kernel.org/bpf/20211028014428.rsuq6rkfvqzq23tg@apollo.localdomain Yes, identical I think. > > I think this might be a bug in the BTF generation, since there should > only be one BTF ID for a type, either in vmlinux or the module BTF. > Maybe Andrii would be able to confirm. Had to put out some fires last week. I chased this down a bit today and best I can tell was the `nf_conn` definitions in BTF were all slightly different. For example, here were the 3 definitions in nf_conntrack.ko alone: [88439] STRUCT 'nf_conn' size=296 vlen=11 'ct_general' type_id=67058 bits_offset=0 'lock' type_id=373 bits_offset=64 'timeout' type_id=160 bits_offset=576 'tuplehash' type_id=67235 bits_offset=640 'status' type_id=1 bits_offset=1536 'ct_net' type_id=4298 bits_offset=1600 '__nfct_init_offset' type_id=4213 bits_offset=1664 'master' type_id=88438 bits_offset=1664 'mark' type_id=67192 bits_offset=1728 'ext' type_id=67236 bits_offset=1792 'proto' type_id=67234 bits_offset=1856 [90882] STRUCT 'nf_conn' size=296 vlen=11 'ct_general' type_id=67058 bits_offset=0 'lock' type_id=373 bits_offset=64 'timeout' type_id=160 bits_offset=576 'tuplehash' type_id=67235 bits_offset=640 'status' type_id=1 bits_offset=1536 'ct_net' type_id=90574 bits_offset=1600 '__nfct_init_offset' type_id=4213 bits_offset=1664 'master' type_id=90881 bits_offset=1664 'mark' type_id=67192 bits_offset=1728 'ext' type_id=67236 bits_offset=1792 'proto' type_id=67234 bits_offset=1856 [92469] STRUCT 'nf_conn' size=296 vlen=11 'ct_general' type_id=67058 bits_offset=0 'lock' type_id=373 bits_offset=64 'timeout' type_id=160 bits_offset=576 'tuplehash' type_id=67235 bits_offset=640 'status' type_id=1 bits_offset=1536 'ct_net' type_id=92160 bits_offset=1600 '__nfct_init_offset' type_id=4213 bits_offset=1664 'master' type_id=92468 bits_offset=1664 'mark' type_id=67192 bits_offset=1728 'ext' type_id=67236 bits_offset=1792 'proto' type_id=67234 bits_offset=1856 Note how `master` and `ct_net` all have different BTF IDs. Best I can tell is that there's some kind of subtle difference in BTF types and it's confusing the dedup algorithm. I went and upgraded to latest pahole (built from today's source tree) to chase the issue down further but the problem went away. Figured I'd write this up in case someone stumbles onto this in the future. [...] Thanks, Daniel
