On 1/22/25 4:35 AM, Anton Gavriliuk wrote:
For storage server solutions space saving is very useful, both
compression and deduplication are important. Deduplication is just a
part of the story.
What if the compression algorithm (lz4 or zstd) could be selected per
device basis with default lz4 ?
It also may not be a bad idea to add compression levels from --fast to --best.
Combining above, users could select different algorithms (lz4, zstd),
different compression ratios and speed for each device.
For instance - there are 2 volumes, one with hot data and lz4, another
with cold data with zstd, but both volumes with deduplication.
Switching algorithms after data has been written to the volume could make older compressed data unreadable, so we don't want to allow that.
Yes, there must be a warning, that after selecting the algorithm there
is no turning back.
An example - nearly a year ago, HPE's software defined storage
solution - "Alletra MP" compression algorithm was switched to lz4 ->
zstd.
If still keeps a single compression algorithm, zstd almost always
looks better lz4 (probably except decompress speed)
We cannot remove support for lz4. There are existing users that rely on it.
I have no objection to adding more options, though. It's on our roadmap
already, so perhaps we'll look into it. If you wanted to put together
patches for adding this, we'd certainly take a look at them.
Matt
Anton
ср, 22 янв. 2025 г. в 04:59, Matthew Sakai <msakai@xxxxxxxxxx>:
It's a reasonable idea. We've discussed it before but it hasn't been
particularly high priority. Since VDO is mainly focused on
deduplication, it's not clear how much it would help to use a different
compression algorithm, but it shouldn't hurt anything, either.
To make this work we'd want to add a new create-time option to specify
the compression algorithm to use. Switching algorithms after data has
been written to the volume could make older compressed data unreadable,
so we don't want to allow that.
Matt
On 1/18/25 12:53 PM, Anton Gavriliuk wrote:
What if zstd can be added to VDO as a compression algorithm along with lz4 ?
It could provide more flexibility for users - faster compression or
better compression ratio.
Anton
