Re: A script for hybrid encryption with openssl

[Nick <oinksocket@xxxxxxxxxxxxxxx>]

On 17/12/2018 22:02, Jakob Bohm via openssl-users wrote:

A simpler way is to realize that the formats used by SMIME/CMS (specifically
the PKCS#7 formats) allow almost unlimited file size, and any 2GiB limit is
probably an artifact of either the openssl command line tool or some of the
underlying OpenSSL libraries.

Yes. I started using openssl's smime implementation, then backed out when I realised there were indeed limits - apparently in the underlying libraries.

On decrypting I got the same kind of errors described in this bug report thread (and elsewhere if you search, but this is the most recent discussion I could find).

"Attempting to decrypt/decode a large smime encoded file created with openssl fails regardless of the amount of OS memory available".
https://mta.openssl.org/pipermail/openssl-dev/2016-August/008237.html

The key points are:

- streaming smime *encryption* has been implemented, but
- smime *decryption* is done in memory, consequentially you can't decrypt anything over 1.5G
- possibly this is related to the BUF_MEM structure's dependency on the size of an int

There's an RT ticket but I could not log in to read this.  But it appears to have been migrated to Git-hub:

https://github.com/openssl/openssl/issues/2515

It's closed - I infer as "won't fix" (yet?) and this is still an issue as my experience suggests, at least in the versions distributed for systems I will be using.

I was using openssl 1.0.2g-1ubuntu4.14 (Xenial) and I've verified it with openssl 1.1.0g-2ubuntu4.3 (Bionic, the latest LTS release fro Ubuntu):

$ openssl version -a
OpenSSL 1.1.0g  2 Nov 2017
built on: reproducible build, date unspecified
platform: debian-amd64
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\""
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"

$ dd if=/dev/zero of=sample.txt count=2M bs=1024
$ openssl req -x509 -nodes -newkey rsa:2048 -keyout mysqldump-secure.priv.pem -out mysqldump-secure.pub.pem
$ openssl smime -encrypt -binary -text -aes256 -in sample.txt -out sample.txt.enc -outform DER -stream mysqldump-secure.pub.pem
$ openssl smime -decrypt -binary -inkey mysqldump-secure.priv.pem -inform DEM -in sample.txt.enc -out sample.txt.restored

Error reading S/MIME message
139742630175168:error:07069041:memory buffer routines:BUF_MEM_grow_clean:malloc failure:../crypto/buffer/buffer.c:138:
139742630175168:error:0D06B041:asn1 encoding routines:asn1_d2i_read_bio:malloc failure:../crypto/asn1/a_d2i_fp.c:191

Anyway, setting up an alternative data format might be suitable if combined
with other functionality requiring chunking, such as recovery from
lost/corrupted data "blocks" (where each block is much much larger than
a 1K "disk block").

I should add that I don't really care about the format, or even the use of openssl - just the ability to tackle large files with the benefits of public key encryption, in a self-contained way without needing fiddly work deploying the keys (as GnuPG seems to require for its keyring, judging from my experience deploying Backup-Ninja / Duplicity using Ansible.)  So other solutions, if tried and tested, might work for me.

Cheers,

Nick

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users