On Tue, Jun 18, 2019 at 01:19:42PM +0200, Arnd Bergmann wrote: > The same bug that gcc hit in the past is apparently now showing > up with clang, which decides to inline __serpent_setkey_sbox: > > crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=] > > Marking it 'noinline' reduces the stack usage from 2112 bytes to > 192 and 96 bytes, respectively, and seems to generate more > useful object code. > > Fixes: c871c10e4ea7 ("crypto: serpent - improve __serpent_setkey with UBSAN") > Signed-off-by: Arnd Bergmann <arnd@xxxxxxxx> > --- > v2: style improvements suggested by Eric Biggers > --- > crypto/serpent_generic.c | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/crypto/serpent_generic.c b/crypto/serpent_generic.c > index e57757904677..56fa665a4f01 100644 > --- a/crypto/serpent_generic.c > +++ b/crypto/serpent_generic.c > @@ -225,7 +225,13 @@ > x4 ^= x2; \ > }) > > -static void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2, u32 r3, u32 r4, u32 *k) > +/* > + * both gcc and clang have misoptimized this function in the past, > + * producing horrible object code from spilling temporary variables > + * on the stack. Forcing this part out of line avoids that. > + */ > +static noinline void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2, > + u32 r3, u32 r4, u32 *k) > { > k += 100; > S3(r3, r4, r0, r1, r2); store_and_load_keys(r1, r2, r4, r3, 28, 24); > -- > 2.20.0 > Reviewed-by: Eric Biggers <ebiggers@xxxxxxxxxx> - Eric