On Tue, Apr 04, 2023 at 11:02:33PM +0000, Viken Kondakji wrote:
> I am using EVP_PKEY_encrypt, and running on a little-endian host using
> RSA public key. Will the binary output of my encryption be in
> big-endian?
Your mistake is using EVP_PKEY_encrypt() directly, rather than a high
level primitive, like CMS_EncryptedData_encrypt().
> The target destination wants the binary data in big-endian and then
> encoded to base 64.
If they are looking for raw RSA signatures, they too are misguided.
> A man of EVP_PKEY_encrypt(3) on my host does not mention any
> endianness.
You shouldn't be using this function directly. It is only for use in
higher-level protocols. Naive misuse inevitably leads to significant
security issues.
--
Viktor.
