Hi,
I occasionally get spurious errors in my ECDSA signatures, and it
appears that when the top byte is over 0x80 of either the R or S
component, that I get a zero pad. I noticed all this when reading
through the source, their was some comments (see below). I noticed a
d2i_ASN1_UINTEGER, but I can't find a coresponding i2d_ version to
create it. The zero pad seems to be the correct behavior, but it seems
to be breaking things.
I understand the ECDSA signature to be a der encoded:
0x30 (ASN1 sequence>
0x<len> (byte len of sequence)
0x20 <ASN1 int header>
0x <int len>
<R>
0x20 <ASN1 int header>
0x <int len>
<S>
Below we can see the zero pad on R where the first byte is 0xB2.
pSignature is what's in question:
26: C_Sign
2019-08-06 02:51:15.252
[in] hSession = 0x100000000000000
[in] pData[ulDataLen] 00007ffe2dcc7520 / 32
00000000 A7 84 A1 44 7F 22 B3 52 98 D8 02 30 E3 7D 21 BD ...D^?".R...0.}!.
00000010 C9 78 3E 3B 9C 1B 53 BA FD 95 B7 93 A6 64 E5 4E .x>;..S......d.N
[out] pSignature[*pulSignatureLen] 0000558c88c9b640 / 71
00000000 30 45 02 21 00 B2 F0 60 57 11 B3 63 83 94 FB 15 0E.!...`W..c....
00000010 74 DC 8C DF A8 2E D9 29 35 89 F7 15 74 70 11 9D t......)5...tp..
00000020 3A E1 3A BE 10 02 20 42 4A 99 54 2E 0D D5 32 92 :.:... BJ.T...2.
00000030 3B 7B 96 E9 2F B8 8B 24 77 38 2F 37 FD 13 98 35 ;{../..$w8/7...5
00000040 4E FC C1 D2 80 6E 46 N....nF
Returned: 0 CKR_OK
-- details and links --
This is the link to the issue request I got filed for more details:
https://github.com/tpm2-software/tpm2-pkcs11/issues/277
https://docs.huihoo.com/doxygen/openssl/1.0.1c/a__int_8c_source.html
95 * Positive integers are no problem: they are almost the same as the DER
96 * encoding, except if the first byte is >= 0x80 we need to add
a zero pad.
266 /* This is a version of d2i_ASN1_INTEGER that ignores the sign bit of
267 * ASN1 integers: some broken software can encode a positive INTEGER
268 * with its MSB set as negative (it doesn't add a padding zero).
269 */
270
271 ASN1_INTEGER *d2i_ASN1_UINTEGER(ASN1_INTEGER **a, const unsigned
char **pp,
272 long length)
273 {
https://www.openssl.org/docs/man1.1.0/man3/d2i_ASN1_UINTEGER.html
