On Wed, Jun 28, 2023 at 04:03:39PM -0700, Krister Johansen wrote:
> commit 0108a4e9f3584a7a2c026d1601b0682ff7335d95 upstream.
>
> When subprograms are in use, the main program is not jit'd after the
> subprograms because jit_subprogs sets a value for prog->bpf_func upon
> success. Subsequent calls to the JIT are bypassed when this value is
> non-NULL. This leads to a situation where the main program and its
> func[0] counterpart are both in the bpf kallsyms tree, but only func[0]
> has an extable. Extables are only created during JIT. Now there are
> two nearly identical program ksym entries in the tree, but only one has
> an extable. Depending upon how the entries are placed, there's a chance
> that a fault will call search_extable on the aux with the NULL entry.
>
> Since jit_subprogs already copies state from func[0] to the main
> program, include the extable pointer in this state duplication.
> Additionally, ensure that the copy of the main program in func[0] is not
> added to the bpf_prog_kallsyms table. Instead, let the main program get
> added later in bpf_prog_load(). This ensures there is only a single
> copy of the main program in the kallsyms table, and that its tag matches
> the tag observed by tooling like bpftool.
>
> Cc: stable@xxxxxxxxxxxxxxx
> Fixes: 1c2a088a6626 ("bpf: x64: add JIT support for multi-function programs")
> Signed-off-by: Krister Johansen <kjlx@xxxxxxxxxxxxxxxxxx>
> Acked-by: Yonghong Song <yhs@xxxxxx>
> Acked-by: Ilya Leoshkevich <iii@xxxxxxxxxxxxx>
> Tested-by: Ilya Leoshkevich <iii@xxxxxxxxxxxxx>
> Link: https://lore.kernel.org/r/6de9b2f4b4724ef56efbb0339daaa66c8b68b1e7.1686616663.git.kjlx@xxxxxxxxxxxxxxxxxx
> Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
> Signed-off-by: Krister Johansen <kjlx@xxxxxxxxxxxxxxxxxx>
> ---
> kernel/bpf/verifier.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 4f2271f27a1d..a89cd34eb5d4 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -12588,9 +12588,10 @@ static int jit_subprogs(struct bpf_verifier_env *env)
> }
>
> /* finally lock prog and jit images for all functions and
> - * populate kallsysm
> + * populate kallsysm. Begin at the first subprogram, since
> + * bpf_prog_load will add the kallsyms for the main program.
> */
> - for (i = 0; i < env->subprog_cnt; i++) {
> + for (i = 1; i < env->subprog_cnt; i++) {
> bpf_prog_lock_ro(func[i]);
> bpf_prog_kallsyms_add(func[i]);
> }
> @@ -12615,6 +12616,9 @@ static int jit_subprogs(struct bpf_verifier_env *env)
>
> prog->jited = 1;
> prog->bpf_func = func[0]->bpf_func;
> + prog->jited_len = func[0]->jited_len;
This 'prog->jited_len' line wasn't part of the original commit and
appears to have snuck in during manual conflict resolution. Let me
clean up and resubmit. Apologies.
> + prog->aux->extable = func[0]->aux->extable;
> + prog->aux->num_exentries = func[0]->aux->num_exentries;
> prog->aux->func = func;
> prog->aux->func_cnt = env->subprog_cnt;
> bpf_prog_jit_attempt_done(prog);
-K
