On Thu, Apr 02, 2020 at 11:23:12PM +0200, Toke Høiland-Jørgensen wrote: > Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> writes: > > > On Fri, Mar 27, 2020 at 12:11:15PM +0100, Toke Høiland-Jørgensen wrote: > >> > >> Current code is in [0], for those following along. There are two bits of > >> kernel support missing before I can get it to where I want it for an > >> initial "release": Atomic replace of the dispatcher (this series), and > >> the ability to attach an freplace program to more than one "parent". > >> I'll try to get an RFC out for the latter during the merge window, but > >> I'll probably need some help in figuring out how to make it safe from > >> the verifier PoV. > > > > I have some thoughts on the second part "ability to attach an freplace > > to more than one 'parent'". > > I think the solution should be more generic than just freplace. > > fentry/fexit need to have the same feature. > > Few folks already said that they want to attach fentry to multiple > > kernel functions. It's similar to what people do with kprobe progs now. > > (attach to multiple and differentiate attach point based on parent IP) > > Similarly "bpftool profile" needs it to avoid creating new pair of fentry/fexit > > progs for every target bpf prog it's collecting stats about. > > I didn't add this ability to fentry/fexit/freplace only to simplify > > initial implementation ;) I think the time had come. > > Yup, I agree that it makes sense to do the same for fentry/fexit. > > > Currently fentry/fexit/freplace progs have single prog->aux->linked_prog pointer. > > It just needs to become a linked list. > > The api extension could be like this: > > bpf_raw_tp_open(prog_fd, attach_prog_fd, attach_btf_id); > > (currently it's just bpf_raw_tp_open(prog_fd)) > > The same pair of (attach_prog_fd, attach_btf_id) is already passed into prog_load > > to hold the linked_prog and its corresponding btf_id. > > I'm proposing to extend raw_tp_open with this pair as well to > > attach existing fentry/fexit/freplace prog to another target. > > Internally the kernel verify that btf of current linked_prog > > exactly matches to btf of another requested linked_prog and > > if they match it will attach the same prog to two target programs (in case of freplace) > > or two kernel functions (in case of fentry/fexit). > > API-wise this was exactly what I had in mind as well. perfect! > > Toke, Andrey, > > if above kinda makes sense from high level description > > I can prototype it quickly and then we can discuss details > > in the patches ? > > Or we can drill further into details and discuss corner cases. > > I have one detail to discuss: What would the bpf_raw_tp_open() call > return on the second attachment? A second reference to the same bpf_link > fd as the initial attachment, or a different link? It's a different link. For fentry/fexit/freplace the link is pair: // target ... bpf_prog (target_prog_fd_or_vmlinux, fentry_exit_replace_prog_fd). So for xdp case we will have: root_link = (eth0_ifindex, dispatcher_prog_fd) // dispatcher prog attached to eth0 link1 = (dispatcher_prog_fd, xdp_firewall1_fd) // 1st extension prog attached to dispatcher link2 = (dispatcher_prog_fd, xdp_firewall2_fd) // 2nd extension prog attached to dispatcher Now libxdp wants to update the dispatcher prog. It generates new dispatcher prog with more placeholder entries or new policy: new_dispatcher_prog_fd. It's not attached anywhere. Then libxdp calls new bpf_raw_tp_open() api I'm proposing above to create: link3 = (new_dispatcher_prog_fd, xdp_firewall1_fd) link4 = (new_dispatcher_prog_fd, xdp_firewall2_fd) Now we have two firewalls attached to both old dispatcher prog and new dispatcher prog. Both firewalls are executing via old dispatcher prog that is active. Now libxdp calls: bpf_link_udpate(root_link, dispatcher_prog_fd, new_dispatcher_prog_fd) which atomically replaces old dispatcher prog with new dispatcher prog in eth0. The traffic keeps flowing into both firewalls. No packets lost. But now it goes through new dipsatcher prog. libxdp can now: close(dispatcher_prog_fd); close(link1); close(link2); Closing (and destroying two links) will remove old dispatcher prog from linked list in xdp_firewall1_prog->aux->linked_prog_list and from xdp_firewall2_prog->aux->linked_prog_list. Notice that there is no need to explicitly detach old dispatcher prog from eth0. link_update() did it while replacing it with new dispatcher prog.
