On Wed, Sep 08, 2021 at 10:19:39AM -0700, Martin KaFai Lau wrote:
> On Wed, Sep 08, 2021 at 09:31:55PM +0800, Hou Tao wrote:
> > Hi,
> >
> > On 9/8/2021 2:06 PM, Martin KaFai Lau wrote:
> > > On Wed, Sep 01, 2021 at 04:53:44PM +0800, Hou Tao wrote:
> > >> Currently if a function ptr in struct_ops has a return value, its
> > >> caller will get a random return value from it, because the return
> > >> value of related BPF_PROG_TYPE_STRUCT_OPS prog is just dropped.
> > >>
> > >> So adding a new flag BPF_TRAMP_F_RET_FENTRY_RET to tell bpf trampoline
> > >> to save and return the return value of struct_ops prog if ret_size of
> > >> the function ptr is greater than 0. Also restricting the flag to be
> > >> used alone.
> > > Thanks for the report and fix! Sorry for the late reply.
> > >
> > > This bug is missed because the tcp-cc func is not always called.
> > > A better test needs to be created to force exercising these funcs
> > > in bpf_test_run(), which can be a follow-up patch in the bpf-next.
> > > Could you help to create this test as a follow up?
> >
> > Yes, will do. The first thought comes into my mind is implementing .get_info hook
> > in a bpf tcp_congestion_ops and checking its return value in userspace by
> > getsockopt(fd, TCP_CC_INFO).
> The bpf-tcp-cc's struct_ops currently does not support ".get_info".
> It will be a good addition also.
>
> Different bpf-tcp-cc implementations have different infos, so it cannot be
> bounded by a fixed struct like 'union tcp_cc_info'. The format should be
> a btf_id followed by the actual info-data. The kernel should be able to
> learn the size of the info-data from the btf_id. The ".get_info" is
> also used by inet_diag for tools (ss) like iproute2. libbpf can pretty-print
> the btf described data and libbpf support is added to iproute2, so pieces
> should be in-place for iproute2's tools to handle data described by btf.
>
> For ".get_info" in getsockopt(TCP_CC_INFO), not sure how the application
> may use them but I think it will at least enable the application log
> them as other kernel's tcp-cc do. The implementation details may
> need some more thoughts but should not be a big issue.
>
> > I also consider to add a new BPF struct_ops
> > for testing purpose, but it may be a little overkill.
> A dummy struct_ops for testing makes sense. It probably should
> be the one done first for testing purpose. Although "get_info"
> is a good add, having a separate testing struct_ops will be easier
> to test other interesting cases in the future.
>
> > I just check that it can be applied both on bpf and bpf-next, do you
> > have other commits in your tree ?
> There is no local commit.
>
> From a quick look, the patch is created from a pretty old tree and it
> is missing the BPF_TRAMP_F_SKIP_FRAME. It is introduced in
typo. I meant missing the BPF_TRAMP_F_IP_ARG.
> commit 7e6f3cd89f04 ("bpf, x86: Store caller's ip in trampoline stack")
> on Jul 15 2021 which is pretty old.
>
> I am only able to apply with the --3way merge like "git am --3way".
> Andrii, is it fine to land the patch like this?
>
> > @@ -1949,17 +1972,19 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
> > struct bpf_tramp_progs *fmod_ret = &tprogs[BPF_TRAMP_MODIFY_RETURN];
> > u8 **branches = NULL;
> > u8 *prog;
> > + bool save_ret;
> >
> > /* x86-64 supports up to 6 arguments. 7+ can be added in the future */
> > if (nr_args > 6)
> > return -ENOTSUPP;
> >
> > - if ((flags & BPF_TRAMP_F_RESTORE_REGS) &&
> > - (flags & BPF_TRAMP_F_SKIP_FRAME))
> > + if (!is_valid_bpf_tramp_flags(flags))
> > return -EINVAL;
> >
> > - if (flags & BPF_TRAMP_F_CALL_ORIG)
> > - stack_size += 8; /* room for return value of orig_call */
> > + /* room for return value of orig_call or fentry prog */
> > + save_ret = flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET);
> > + if (save_ret)
> > + stack_size += 8;
> >
> > if (flags & BPF_TRAMP_F_SKIP_FRAME)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> > /* skip patched call instruction and point orig_call to actual
