On Fri, May 15, 2015 at 09:47:05PM -0700, Tom Marshall wrote:
> On Fri, May 15, 2015 at 06:14:24PM -0700, Jaegeuk Kim wrote:
> > On Thu, May 14, 2015 at 09:50:44AM -0700, Tom Marshall wrote:
> > > Please keep in mind that I'm also working on transparent
> > > compression. I'm watching this thread closely so that I can
> > > implement a compression library alongside the crypto library. If
> > > there is any interest or benefit, I would be glad to work together
> > > so that the two can be done cooperatively at the same time.
> >
> > I can't imagine quickly how compression code can be shared with crypto.
> > The basic approach for compression would be that X pages can be compressed into
> > small number of pages, Y, which can be a X to Y mapping.
> > But, this per-file encryption supports only 1 to 1 4KB mapping, so that it could
> > be quite a simple implementation.
>
> No, I don't intend to share actual code with crypto -- at least not much.
> I'm more interested in looking at how the crypto layer is implemented to
> give me clues about how to implement a compression layer.
Ok, I see.
Currently, I've been writing up fs/crypto having shared crypto codes between
ext4 and f2fs; I refactored existing codes a little bit though.
I'm approaching to introduce a fscypt_operations for each filesystems like this.
struct fscrypt_operations {
int (*get_context)(struct inode *, void *, size_t, void *);
int (*set_context)(struct inode *, const void *, size_t, int, void *);
int (*prepare_new_context)(struct inode *);
bool (*is_encrypted)(struct inode *);
bool (*empty_dir)(struct inode *);
unsigned (*max_namelen)(struct inode *);
};
And, the following two basic functions will be used by filesystems:
fscrypt_encrypt_page() and fscrypt_decrypt_page().
>
> > Could you elaborate on your approach or design? Or, codes?
> > Whatever, IMO, it needs to implement it by any filesystem first.
>
> I don't really have any working code yet. I will probably get to that in
> the coming few weeks. Right now I'm still working with the ugly VFS
> stacking implementation that I posted initially.
>
> The thing that I have done is dismissed the standard compression framing
> formats.
>
> zlib (and gzip) are designed for streaming and it is quite difficult to
> implement random access on it. See the example code in the zlib source,
> zran.c. It's not really tenable because 32kb of prior data is required to
> be kept as priming information. Even doing fully encapsulated blocks with
> Z_FINISH, there is still no way to skip over data without decompressing it
> first to build an index.
>
> lz4 is somewhat better in that blocks are self contained. But block lengths
> must be read sequentially. This means that reading an arbitrary position in
> a file requires a proportional number of reads to find the desired block.
>
> So, I am working with a simple framing format that I threw together. The
> header has a compression method (zlib or lz4), block size, original input
> size, and a block map.
Thank you for sharing the approach, and it makes sense to improve random
read performance.
Thanks,
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
