Re: Bluetooth ECDH selftest failed (endianness issue?)

[Andrey Batyiev <batyiev@xxxxxxxxx>]

Hello Marcel,

> On Sat, Dec 29, 2018 at 9:35 AM Marcel Holtmann <marcel@xxxxxxxxxxxx> wrote:
> I think that our ECDH code was endian safe, but then it got changed at some point to use standard crypto and maybe something went wrong there. Can just provide the btmon -w trace.log for the SMP pairing so that I can have a look at the binary trace.

I found out that if I change "swap_digits" method in
"net/bluetooth/ecdh_helper.c" to

static inline void swap_digits(u64 *in, u64 *out, unsigned int ndigits)
{
int i;

for (i = 0; i < ndigits; i++)
        out[i] = in[ndigits - 1 - i];
}

then BLE pairing on big-endian become operational. I'm not sure what
proper fix should be: is it a problem with crypto API usage or a
problem with crypto itself?

Thanks,
   Andrey