Re: Migrating low level ec APIs from openssl1.1.1 to openssl3.0.7 high level EVP APIs

[Vishal Patil <vishable@xxxxxxxxx>]

Thanks Tomas.

The imported publicKey from octet data doesn't seem to be complete in new implementation ( I tried printing pem key in both cases after import)

In New implementation key is exactly same as input octet data.

however in old implementation (using low level APIs) public key derived from octet data has some more contents along with input octet data.

----------------------------------------------------------------------------------------------------------

New implementation(high level APIs):

In importPublicKey , length=67, data="">

3 0 cc e1 74 f0 ae 79 95 bc 39 62 6d 20 9a b9 88 8c c0 f1 d1 ae 77 44 f4 8f 65 52 25 2c 49 b9 5e 77 af 79 8a d cb a 93 b3 3d 37 b8 8d 4c 71 a6 78 2c 5f c0 52 15 8b 96 c1 5c 0 a4 25 41 a1 8 4d 59

key print(high APIs)...
 Public-Key: (521 bit)
 pub:
     03:00:cc:e1:74:f0:ae:79:95:bc:39:62:6d:20:9a:
     b9:88:8c:c0:f1:d1:ae:77:44:f4:8f:65:52:25:2c:
     49:b9:5e:77:af:79:8a:0d:cb:0a:93:b3:3d:37:b8:
     8d:4c:71:a6:78:2c:5f:c0:52:15:8b:96:c1:5c:00:
     a4:25:41:a1:08:4d:59

----------------------------------------------------------------------------------------------------------

Old implementation (low level APIs):

In importPublicKey , length=67, data="">

2 1 37 88 37 3a 5e 21 19 fe 1 5d 9b 4f 4d 5c 7 7d c4 72 ae 1b 9f 5a 7b 1a 22 9c 21 f cf a5 c3 a9 94 20 e5 8c 72 55 25 6d a8 eb 1c ac d8 d7 12 6b 76 29 f2 ee 2 2f d6 1f e3 b1 20 2a 21 4 98 a1 a5

key print...                                                                                                                                                                     

Public-Key: (521 bit)                                                                                                                                                                   

pub:

04:01:37:88:37:3a:5e:21:19:fe:01:5d:9b:4f:4d:                                                                                                                                            5c:07:7d:c4:72:ae:1b:9f:5a:7b:1a:22:9c:21:0f:                                                                                                                                            cf:a5:c3:a9:94:20:e5:8c:72:55:25:6d:a8:eb:1c:                                                                                                                                            ac:d8:d7:12:6b:76:29:f2:ee:02:2f:d6:1f:e3:b1:                                                                                                                                            20:2a:21:04:98:a1:a5:01:dc:b9:90:ce:1e:14:bb:                                                                                                                                            7d:18:9f:e6:da:ae:03:7d:ca:22:f3:b5:4a:0a:ef:                                                                                                                                            e1:82:a5:a1:61:c2:3a:b1:f8:82:bb:d7:6c:7f:ce:                                                                                                                                            05:e5:26:36:94:0f:d1:ff:17:87:44:a0:45:83:8f:                                                                                                                                           

 d8:bc:b8:e8:88:c2:d4:22:6e:8e:2d:db:b8

If you notice then in old implementation public key has more data appended along with input octet.

And I think this is what missing in my new implementation, all the OSSL params being set basically not having any impact on deriving the key.

Regards,

Vishal

On Thu, 30 Mar 2023, 22:16 Tomas Mraz, <tomas@xxxxxxxxxxx> wrote:

I do not see any issue with the pubkey import.

With the same private key and the same peer pubkey the computed shared
key should be exactly the same - if you drop the eciesKeyDerivationFn
from the legacy implementation it should produce the same shared key as
the new implementation.

Tomas Mraz

On Thu, 2023-03-30 at 21:56 +0530, Vishal Patil wrote:
> Thanks Tomas for pointing that out,  I have tried calling SHA512 on
> raw envelope however issue still persists.
>
> I suspect public key import has issues. Could you please check if all
> necessary evp routines and parameters are present to get the public
> key from octet data?
>
> Thanks 
>
> On Thu, 30 Mar 2023, 12:20 Tomas Mraz, <tomas@xxxxxxxxxxx> wrote:
> > In the legacy method you've supplied the eciesKeyDerivationFn
> > function
> > to the ECDH_compute_key() which is applied after the raw shared
> > secret
> > from the ECDH is computed. The EVP_PKEY_derive() does not do that.
> > Do
> > not be confused with the KDF parameters - they apply only to the
> > X9.63
> > KDF if enabled but that is most probably a completely different KDF
> > from what you implement in the eciesKeyDerivationFn.
> >
> > However that is no problem, you can just apply the
> > eciesKeyDerivationFn
> > on the output of the ECDH yourself. This is how the KDF is called
> > in
> > ECDH_compute_key():
> > https://github.com/openssl/openssl/blob/master/crypto/ec/ec_kmeth.c#L168
> >
> > Tomas Mraz, OpenSSL
> >
> > On Thu, 2023-03-30 at 01:30 +0530, Vishal Patil wrote:
> > > We are having some troubles converting some code from OpenSSL
> > > 1.1.x
> > > to OpenSSL
> > > 3.0.7 APIs, to get rid of deprecation warnings, and hope someone
> > > may
> > > be able to
> > > give us some hints in the right direction.
> > >
> > >
> > > Part1:  We are encrypting some text file with openssl3 commands
> > > and extracting public part(ephdata) of provate key and appending
> > > it
> > > to encryptedfile.
> > > All openssl operations done using openssl(version 3.0.7) commands
> > > in
> > > bash script.
> > >
> > > openssl ecparam -name secp521r1 -genkey -noout -out ephpri.pem
> > > openssl pkeyutl -derive -inkey ephpri.pem -peerkey ecpubkey.pem -
> > > out
> > > derive.bin
> > > ENVELOPE=$(openssl dgst -sha512 derive.bin | awk '{print $2}')
> > > AESKEY=$(echo $ENVELOPE | cut -c1-64)
> > > AESIV=$(echo $ENVELOPE | cut -c65-96)
> > > echo "AESKEY = $AESKEY"
> > > echo "AESIV  = $AESIV"
> > > # encrypt file.txt
> > > openssl enc -e -aes-256-ctr -in file.txt -out file.enc -K $AESKEY
> > > -iv
> > > $AESIV
> > >
> > > #extract and append pub part of private-key
> > > # export public data of the ephemeral key
> > > openssl ec -in ephpri.pem -pubout -out ephdata.tmp -outform DER -
> > > conv_form compressed
> > > OFFSET=$(openssl asn1parse -inform DER -in ephdata.tmp | grep
> > > "BIT
> > > STRING" | awk -F ':' '{print $1}' | awk '{print $1}')
> > > OFFSET=$(expr $OFFSET + 3)
> > > dd if=ephdata.tmp of=ephdata bs=$OFFSET count=1024 skip=1
> > > append_file file.enc ephdata
> > >
> > > Part2: We are processing the encrypted file on other peer using
> > > application which uses openssl3 APIs.
> > > 1) extraction of ephdata(pub part of key in part1) from encrypted
> > > file.
> > > 2) preparing public key from ephdata
> > > 3) prearing hash using own private key and derived public key
> > > 4) deriving of AES keys from hash and decrypting encrypted file
> > > using
> > > the same.
> > >
> > > Low level APIs: ( part2 decryption succeeds):
> > >
> > > group = EC_GROUP_new_by_curve_name(NID_secp521r1);
> > > EC_GROUP_precompute_mult(group, nullptr)
> > > EC_GROUP_set_point_conversion_form(group,
> > > POINT_CONVERSION_COMPRESSED);
> > >
> > > publicKey  = EC_KEY_new();
> > > EC_KEY_set_group(key, group)
> > > point = EC_POINT_new(group);
> > > EC_POINT_oct2point(group, point, data, length, nullptr)  // data
> > > is
> > > ephdata - derived in part1 
> > > EC_KEY_set_public_key( publicKey  , point)
> > > EC_POINT_free(point);
> > > EC_KEY_check_key( publicKey  ) // key is public key derived from
> > > ephdata
> > >
> > > ECDH_compute_key(envelope,
> > >                          SHA512_DIGEST_LENGTH,
> > >                          EC_KEY_get0_public_key(publicKey),
> > >                          privateKey,
> > >                          eciesKeyDerivationFn)
> > > -----------------------------------------------------------------
> > > ----
> > > -----------------------------------------------------------------
> > >
> > > Replaced with high level APIs ( part2 decryption fails) :
> > >    
> > > // ctx setting with named curve
> > >  nid = NID_secp521r1;
> > >     pctx = EVP_PKEY_CTX_new_from_name(NULL, "EC", NULL);
> > >     EVP_PKEY_keygen_init(pctx)
> > >     EVP_PKEY_CTX_set_ec_paramgen_curve_nid(pctx, nid)
> > >     params[0] = OSSL_PARAM_construct_utf8_string("point-format",
> > > compressionState, 0);
> > >     params[1] = OSSL_PARAM_construct_end();
> > >     EVP_PKEY_CTX_set_params(pctx, params)
> > >
> > > // public key derivation from pubdata from peer
> > >    dctx = EVP_PKEY_CTX_new_from_name(NULL, "EC", NULL)
> > >    EVP_PKEY_fromdata_init(dctx)
> > >     //param struct for fromdata
> > >     OSSL_PARAM params[4];
> > >     char *secp521r1 = (char *)"secp521r1";
> > >     params[0] = OSSL_PARAM_construct_octet_string("pub", (void *)
> > > data, length);
> > >     params[1] = OSSL_PARAM_construct_utf8_string("group",
> > > secp521r1,
> > > 0);
> > >     char *compressionState = (char *)"compressed";
> > >     params[2] = OSSL_PARAM_construct_utf8_string("point-format",
> > > compressionState, 0);
> > >     params[3] = OSSL_PARAM_construct_end();
> > >     EVP_PKEY * public_key = nullptr;
> > >     EVP_PKEY_fromdata(dctx, &public_key   , EVP_PKEY_PUBLIC_KEY,
> > > params) 
> > >
> > > // hash/envelope derivation 
> > > dctx = EVP_PKEY_CTX_new_from_pkey(NULL, d->privateKey, NULL);
> > > EVP_PKEY_derive_init(dctx)
> > > OSSL_PARAM params[4];
> > > unsigned int padding = 1;
> > > params[0] = OSSL_PARAM_construct_uint(OSSL_EXCHANGE_PARAM_PAD,
> > > &padding);
> > > params[1] =
> > > OSSL_PARAM_construct_utf8_string(OSSL_EXCHANGE_PARAM_KDF_DIGEST,
> > >                         (char*)"sha512", 0);
> > > params[2] =
> > > OSSL_PARAM_construct_size_t(OSSL_EXCHANGE_PARAM_KDF_OUTLEN,
> > >                         &length);
> > > params[3] = OSSL_PARAM_construct_end();
> > > EVP_PKEY_CTX_set_params(dctx, params)
> > > EVP_PKEY_derive_set_peer(dctx, publicKey)
> > > EVP_PKEY_derive(dctx, envelope, &length)
> > >
> > >
> > > Issue:
> > > If part2 application uses high level APIs, decryption of the file
> > > fails.
> > > However if part2 application uses low level APIs ( deprecated in
> > > openssl3) decryption succeeds.
> > > Issue could be either in publickey derivation from ephdata or
> > > hash.
> > >
> > >
> > > Thanks,
> > >
> >

--
Tomáš Mráz, OpenSSL