Re: [RFC] Snappy compressor for Linux Kernel (specifically, zram)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Apr 15, 2011 at 05:21, Greg KH <greg@xxxxxxxxx> wrote:
> Why is this needed to be added to the kernel?  What does it provide that
> users or other parts of the kernel needs?

It is functionally a general data compression tool that trades off
compression ratio for speed. It is optimized for x86-64 and there
achieves compression at 250MB/sec and decompression at 500MB/sec
(YMMV*). This is a better mousetrap, that can and should replace LZO
in every place where the kernel currently uses LZO.

Compressing the kernel image, initial ram drive, etc. are all in the
future but currently, the best way to use a very fast compressor is
with zram: it enables running jobs with worksets that don't fit in RAM
with reasonable performance. For example, gcc 4.6.0 is close to having
usable link time optimization**, but for large program (such as gcc)
it needs a lot of RAM. Swap to zram lets you run it on memory
constrained machines.

It should not replace LZO where it is not faster. I have no benchmark
data from non-x86 hardware (I have used qemu to make sure it does work
on non-x86, though), and it's possible it would be slow on platforms
that do not have fast unaligned access, fast 64 bit, are not little
endian, or have slow 32bit multiplication. If that turns out to be
true, maybe we can alter it to stay compatible with the bitstream
format*** and be faster on those platforms, but for that to happen I
need interest from people with access to the hardware.

I have just learned of the existence of something called zCache. It
seems like a likely candidate for the next step.

The original code has been used in production by Google since 2005 or
so. It is stable and secure. Unless I've ruined it somehow when
porting, of course.

* - https://github.com/zeevt/csnappy/raw/master/userspace_benchmark.txt
    https://github.com/zeevt/csnappy/raw/master/zram_benchmark.txt
** - one showstopper remaining:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48207
*** - I have written a description of the format in English and C :
https://github.com/zeevt/csnappy/blob/master/csnappy_simple.c

> As for the right list, we can help you out, but you have read
> Documentation/SubmittingPatches and Documentation/CodingStyle in the
> kernel tree already, right?

I have read SubmittingPatches and CodingStyle.
I have fixed the things about which checkpatch has complained.

I'll send the patches in proper form shortly.

> thanks,
>
> greg k-h

Thank you,
-Z.T.
_______________________________________________
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxx
http://driverdev.linuxdriverproject.org/mailman/listinfo/devel



[Index of Archives]     [Linux Driver Backports]     [DMA Engine]     [Linux GPIO]     [Linux SPI]     [Video for Linux]     [Linux USB Devel]     [Linux Coverity]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux