On Sat, 30 Aug 2008, Herbert Xu wrote: > On Fri, Aug 29, 2008 at 01:41:59PM +0000, Geert Uytterhoeven wrote: > > From: Geert Uytterhoeven <Geert.Uytterhoeven@xxxxxxxxxxx> > > > > Add a (de)compression module for the "zlib" format using the crypto API. > > I think we can safely conclude that our current compression > interface sucks for what you're trying to achieve :) > > > While both the "zlib" and "deflate" crypto modules are implemented on top of > > the zlib library, they differ in the following aspects: > > - The "deflate" crypto module (used by IPSec and UBIFS) does not support > > partial decompression, i.e. all compressed data has to be passed at once. > > The "zlib" crypto module does support partial decompression; > > zlib_decompress() will return -EAGAIN if not all compressed data has been > > passed. > > - The deflate crypto module uses the raw deflate data format (zlib is > > initialized with a windowBits parameter of -DEFLATE_DEF_WINBITS = -11), > > while e.g. squashfs and axfs use the zlib data format, with the default > > windowBits parameter DEF_WBITS = 15. > > Both parameters are incompatible with each other due to the different data > > formats, as indicated by the sign of the windowbits parameter. > > The absolute value of this parameter is the base two logarithm of the > > maximum window size, and larger values are backwards compatible with > > smaller values (as far as decompression is concerned). > > Therefore I suggest that we change the interface rather than > mutilate the implementations. > > So here are a few things we should add: > > 1) Separate alloc functions for comp/decomp to avoid the memory > wastage you've identified. > > 2) Provide parameters to these alloc functions through an opaque > pointer. The format of the paramters will be determined by the > name of the algorithm, i.e., if you want to change the parameters > then you should change the name as well (e.g., deflate => deflate2). > > This removes the need to dupliate the implementation just because > you want 15 instead of -11. > > 3) Provide init/update/final (one set each for comp and decomp) > functions similar to crypto_hash, in addition to the current > comp/decomp functions. Thanks for your suggestions! I started tackling #3 (init/update/final), as that looks like the most complicated one. As a proof-of-concept, I'm sending the following 4 patches, which gives a working version of SquashFS using the CRYPTO API for decompression: - [1/4] crypto: Add partial decompression support to the CRYPTO API. - [2/4] crypto: Add a "zlib" crypto module - [3/4] crypto: Add a test for the "zlib" crypto module - [4/4] squashfs: Make SquashFS 4 use the new "zlib" crypto module As pointed out in patch [1/4], some underlying (de)compression implementations do not support partial (de)compression. This is the case for e.g. LZO, which is already supported by a crypto module. While one-shot (de)compression can easily be implemented on top of a partial (de)compression API, the inverse is not true. Hence there should be a way to ask for a "one-shot" vs. "partial" decompression module at crypto_alloc_comp() time. Do you have any suggestions how to handle this? Perhaps by using "oneshot(%s)" and "partial(%s)", cfr. what is done in other parts of the CRYPTO API where orthogonal features are combined? For #1 (Separate alloc functions for comp/decomp), I'm wondering if a similar scheme be used? "comp(%s)", "decomp(%s)", "both(%s)" (any better name?)? Or crypto_alloc_comp_only(), crypto_alloc_decomp_only(), and crypto_alloc_comp()? Or would it be better to use the CRYPTO_ALG_TYPE_* flags in crypto_alloc_comp()? I'm still a bit puzzled how exactly everything fits together in the different layers of the CRYPTO API. Thanks for your comments! With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: Geert.Uytterhoeven@xxxxxxxxxxx Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010 -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
