On Fri, May 29, 2020 at 06:12:53PM +0200, Peter Zijlstra wrote:
> On Fri, May 29, 2020 at 11:05:14AM -0500, Josh Poimboeuf wrote:
>
> > It looks to me like GCC is doing the right thing. That likely()
> > translates to:
> >
> > # define likely(x) (__branch_check__(x, 1, __builtin_constant_p(x)))
> >
> > which becomes:
> >
> > #define __branch_check__(x, expect, is_constant) ({ \
> > long ______r; \
> > static struct ftrace_likely_data \
> > __aligned(4) \
> > __section(_ftrace_annotated_branch) \
> > ______f = { \
> > .data.func = __func__, \
> > .data.file = __FILE__, \
> > .data.line = __LINE__, \
> > }; \
> > ______r = __builtin_expect(!!(x), expect); \
> > ftrace_likely_update(&______f, ______r, \
> > expect, is_constant); \
> > ______r; \
> > })
> >
> > Here 'x' is the call to user_access_begin(). It evaluates 'x' -- and
> > thus calls user_access_begin() -- before the call to
> > ftrace_likely_update().
> >
> > So it's working as designed, right? The likely() just needs to be
> > changed to likely_notrace().
>
> But if !x (ie we fail user_access_begin()), we should not pass STAC() on
> the way to out_err. OTOH if x, we should not be jumping to out_err.
>
> I'm most confused... must not stare at asm for a while.
Yeah, I saw that call to ftrace_likely_update() and got distracted. I
forgot it's on the uaccess safe list.
>From staring at the asm I think the generated code is correct, it's just
that the nested likelys with ftrace profiling cause GCC to converge the
error/success paths. But objtool doesn't do register value tracking so
it's not smart enough to know that it's safe.
The nested likelys seem like overkill anyway -- user_access_begin() is
__always_inline and it already has unlikely(), which should be
propagated.
So just remove the outer likelys?
diff --git a/arch/x86/lib/csum-wrappers_64.c b/arch/x86/lib/csum-wrappers_64.c
index a12b8629206d..ee63d7576fd2 100644
--- a/arch/x86/lib/csum-wrappers_64.c
+++ b/arch/x86/lib/csum-wrappers_64.c
@@ -27,7 +27,7 @@ csum_and_copy_from_user(const void __user *src, void *dst,
might_sleep();
*errp = 0;
- if (!likely(user_access_begin(src, len)))
+ if (!user_access_begin(src, len))
goto out_err;
/*
@@ -89,7 +89,7 @@ csum_and_copy_to_user(const void *src, void __user *dst,
might_sleep();
- if (unlikely(!user_access_begin(dst, len))) {
+ if (!user_access_begin(dst, len)) {
*errp = -EFAULT;
return 0;
}
