Hi Linus,
On 2023/6/29 04:10, Linus Torvalds wrote:
On Wed, 28 Jun 2023 at 11:34, David Howells <dhowells@xxxxxxxxxx> wrote:
What about something like the Intel on-die accelerators (e.g. IAA and QAT)? I
think they can do async compression.
I'm sure they can. And for some made-up benchmark it might even help.
Do people use it in real life?
The *big* wins come from being able to do compression/encryption
inline, when you don't need to do double-buffering etc.
Anything else is completely broken, imnsho. Once you need to
double-buffer your IO, you've already lost the whole point.
I'm not sure if I could say much about this for now, yet we're
slowly evaluating Intel IAA builtin DEFLATE engine for our
Cloud workloads and currently we don't have end-to-end numbers
yet.
Storage inline accelerators are great, especially
"do {en,de}cryption inline" since it consumes very little
on-chip memory, yet afaik "(de)compression" inline engine story
is different since it needs more SRAM space for their LZ sliding
windows for matching (e.g. 32kb for deflate each channel, 64kb
for LZ4 each channel, and much much more for Zstd, LZMA, etc.
I think those are quite expensive to integrate) in addition to
some additional memory for huffman/FSE tables.
So in production, inline "(de)compression" accelerators are
hardly seen as a part of storage at least in end consumer
markets.
Intel already has their on-die accelerators (IAA very recently),
yeah, it still needs double-buffer I/O, but we're considering
at least using in async writeback/readahead use cases as a
start for bulk async I/Os, which are not quite latency
sensitive. Intel also shows their Zswap work [1] , yet I don't
dive into that since I'm only focusing on storage use cases.
As for crypto current apis (no matter acomp or scomp), I'm not
sure I will say more too since I mostly agree with what Eric
said previously.
Thanks,
Gao Xiang
[1] https://lore.kernel.org/r/20230605201536.738396-1-tom.zanussi@xxxxxxxxxxxxxxx/
Linus
