On Thu, Jun 18, 2020 at 05:10:08PM +0200, Ronny Meeus wrote: > > What is the "endianness" of this CPU? Is it big-endian or > > little-endian? And does OpenSSL expect the correct one? > > This is a big endian CPU. And, is OpenSSL configured for the correct endianness? On Thu, Jun 18, 2020 at 05:06:50PM +0200, Ronny Meeus wrote: > I instrumented both the "probable_prime" and the "bnrand" functions as > you can see below and the values actually change (see below). > But what is strange is that the value of "rnd->size" is only 24 and > the size of the sizeof(rnd->d) is 4 bytes = 32 bits which results only If rnd->d is a pointer, you want sizeof(rnd->d[0]). Or if you literrally mean "sizeof(sizeof(rnd->d))" than that's expected with a 32-bit address space. Either way, you're perhaps reporting a misleading value. > So there is a problem with the calculation of the size. Or not. > It might have to do with the fact that we run on a 64b kernel with a > 32b user land and that we use only 1 toolchain to compile both the > kernel and the user-land. The size of the address space (size of pointers) should not matter, what matters are the sizes "int", "long", "long long", ... > probable_prime 0 loop=228 again=114 > bnrand bits=1536 top=1 bottom=1 bytes=192 > data: ff ec 15 94 95 10 ca 68 0f 54 06 0f b9 14 bc da 5d 33 58 4e 3e ---- 5 --- > 35 ca d5 30 c8 33 a5 c1 61 7f d6 00 26 98 45 df 73 34 3c 32 74 89 e2 > da 03 72 20 fe 50 23 85 e7 a0 a6 97 27 38 e8 3e 4d 1d d6 11 66 93 d2 > 01 40 5a 9e 23 31 1f a8 a8 7c 46 c9 eb 79 02 44 9d 9c a7 fe 5e 80 f4 > 0b 68 ec a5 2f 59 59 a3 02 56 6b 16 7a 01 a5 bb 3c c9 28 71 3b f5 44 > 30 a3 00 fb f0 de 65 2a 28 8d 8b 8d 2c 71 84 65 33 a9 73 8d 4e ce d4 > 86 ce 1e 6d d7 2f a4 2d 6c 1c 27 88 0e 2a 0c 09 d0 ff 10 e7 55 7b 55 > a9 34 5c 36 88 c9 03 64 f2 ac ae cc 64 f7 a2 3c 6d 62 72 55 8d 99 69 ---- 4 --- ---- 3 --- ---- > 20 2e 33 8b b6 21 20 dd 57 ef 2 --- ---- 1 --- > probable_prime BN_num_bits=1473 bits=1536 > BN_num data (size=24 top=24 neg=0 flags=13, sizeof(*d)=4): > 20dd57f1 9969202e f7a23c6d c90364f2 [...] 9510ca68 -- 1 -- -- 2 -- -- 3 -- -- 4 -- ... -- 5 -- However, clearly the hex output above captures only every other 32-bits of the actual byte array. Perhaps there's something wrong with your compilation tool-chain. -- Viktor.