On Tue, Jul 30, 2024 at 6:44 PM Emil Velikov <emil.l.velikov@xxxxxxxxx> wrote:
>
> On Tue, 30 Jul 2024 at 01:51, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> >
> > On Sat, Jul 27, 2024 at 12:18 AM Emil Velikov <emil.l.velikov@xxxxxxxxx> wrote:
> > >
> > > Hi Sedat, all,
> > >
> > > The mixed quoting is confusing me a bit, hopefully not to the point
> > > where I miss some questions.
> > >
> > > On Thu, 25 Jul 2024 at 21:00, Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > > >
> > > > On Thu, Jul 25, 2024 at 9:23 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > > > >
> > > > > dileks@iniza:~/src/linux/git$ git grep zstd scripts/ | egrep '\-(19|22|T0)'
> > > > > scripts/Makefile.lib: cmd_zstd = cat $(real-prereqs) | $(ZSTD) -19 > $@
> > > > > scripts/Makefile.lib: cmd_zstd22 = cat $(real-prereqs) | $(ZSTD)
> > > > > -22 --ultra > $@
> > > > > scripts/Makefile.lib: cmd_zstd22_with_size = { cat
> > > > > $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@
> > > > > scripts/Makefile.modinst: cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> > > > >
> > > > > -sed@-
> > > > >
> > > > > On Thu, Jul 25, 2024 at 9:09 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am a big fan of ZSTD - here version 1.5.6 on Debian/unstable AMD64.
> > > > > >
> > > > > > When playing with CONFIG_MODULE_COMPRESS_ZSTD=y I observed a 35MiB
> > > > > > greater linux-image Debian file.
> > > > > >
> > > > > > Debian: ~100MiB <--- CONFIG_MODULE_COMPRESS_XZ=y
> > > > > > Selfmade: ~135MiB <--- CONFIG_MODULE_COMPRESS_ZSTD=y
> > > > > >
> > > > > > So, I checked the used compression-level of ZSTD in the build-process:
> > > > > >
> > > > > > [ vmlinux.bin.zst ]
> > > > > > $ grep zstd build-log_6.10.1-1-amd64-clang18-kcfi.txt | grep vmlinux.bin.zst
> > > > > > { cat arch/x86/boot/compressed/vmlinux.bin
> > > > > > arch/x86/boot/compressed/vmlinux.relocs | zstd -22 --ultra; printf
> > > > > > \370\040\157\003; } > arch/x86/boot/compressed/vmlinux.
> > > > > > bin.zst
> > > > > >
> > > > > > [ amdgpu.ko.zst ]
> > > > > > $ grep zstd build-log_6.10.1-1-amd64-clang18-kcfi.txt | grep '\-T0' |
> > > > > > grep amdgpu
> > > > > > zstd -T0 --rm -f -q
> > > > > > debian/linux-image-6.10.1-1-amd64-clang18-kcfi/lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko
> > > > > >
> > > > > > Here some numbers:
> > > > > >
> > > > > > [ Values in KiB ]
> > > > > > $ du -k amdgpu.ko*
> > > > > > 29264 amdgpu.ko
> > > > > > 5672 amdgpu.ko.zst <--- XXX: -T0
> > > > > > 4264 amdgpu.ko.zst_T0-19 <--- XXX: -T0 -19
> > > > > > 4260 amdgpu.ko.zst_T0-22
> > > > > > 4308 amdgpu.ko.zst_T0-22-ultra
> > > > > >
> > > > > > $ time zstd -T0 -v amdgpu.ko
> > > > > > *** Zstandard CLI (64-bit) v1.5.6, by Yann Collet ***
> > > > > > Note: 2 physical core(s) detected
> > > > > > zstd: amdgpu.ko.zst already exists; overwrite (y/n) ? y
> > > > > > amdgpu.ko : 19.38% ( 28.6 MiB => 5.54 MiB, amdgpu.ko.zst)
> > > > > >
> > > > > > real 0m2,496s
> > > > > > user 0m0,448s
> > > > > > sys 0m0,064s
> > > > > >
> > > > > > $ time zstd -T0 -19 -v amdgpu.ko
> > > > > > *** Zstandard CLI (64-bit) v1.5.6, by Yann Collet ***
> > > > > > Note: 2 physical core(s) detected
> > > > > > amdgpu.ko : 14.56% ( 28.6 MiB => 4.16 MiB, amdgpu.ko.zst)
> > > > > >
> > > > > > real 0m22,821s
> > > > > > user 0m22,759s
> > > > > > sys 0m0,080s
> > > > > >
> > > > > > My wish is to use the zstd-option "-T0" (see vmlinux.bin.zst) everywhere.
> > > > > >
> > > > > > For CONFIG_MODULE_COMPRESS_ZSTD=y: Add compress-level = -19
> > > > > >
> > > > > > NOTE: -22 w/ or w/o ultra has no big effect here and increases even
> > > > > > more build-time.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Best regards,
> > > > > > -Sedat-
> > > >
> > > > [ Adding some more folks - kmod / authors of commit c3d7ef377eb2
> > > > ("kbuild: add support for zstd compressed modules") ]
> > > >
> > > > Debian/unstable AMD54 ships kmod version 32+20240611-1.
> > > >
> > > > To use better/higher ZSTD compression-level for
> > > > CONFIG_MODULE_COMPRESS_ZSTD=y - do we need a newer KMOD version?
> > > >
> > > > DIFF for Linux-6.10.1:
> > > >
> > > > $ git diff scripts/Makefile.modinst
> > > > diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> > > > index 0afd75472679..2a8b37c80cbf 100644
> > > > --- a/scripts/Makefile.modinst
> > > > +++ b/scripts/Makefile.modinst
> > > > @@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@
> > > > quiet_cmd_xz = XZ $@
> > > > cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
> > > > quiet_cmd_zstd = ZSTD $@
> > > > - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> > > > + cmd_zstd = $(ZSTD) -T0 -19 --rm -f -q $<
> > > >
> > > > $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> > > > $(call cmd,gzip)
> > > >
> > >
> > > The compression tools (zstd, xz, etc) are used predominantly for two
> > > separate topics - kernel image and kernel module compression.
> > >
> > > Off the top of my head: in the former case, reasonable compression is
> > > used by all tools. Be that zstd, lzo, lzma, you name it.
> > > Honestly I don't know why and what's the merit behind the levels chosen.
> > >
> > > In the modules case, things are also consistent where the default
> > > compression level is used throughout.
> > > Here it makes sense to keep the default since you can consider that
> > > the decompression can be done by either kernel or user-space. Where
> > > each comes with it's limitations:
> > >
> > > The in-kernel one does not support ultra levels for zstd (or it used
> > > to at some point) IIRC. The kmod tools themselves rely on the zstd
> > > library liked in, although if one is using busybox then zstd
> > > compressed modules were not supported last time I checked (a year or
> > > so ago).
> > >
> > > So for example: you can have zstd --22 modules, if the kernel + kmod
> > > are new enough (have in-kernel decompression) things will fail. If
> > > they're not, things will work just fine.
> > >
> > > Which is why, I think it's up-to your distro/builder to choose the
> > > policy. Here is what we've been using In Arch `ZSTD_CLEVEL=19 make ...
> > > modules_install`.
> > >
> > > Tl:Dr: You'd want to use `ZSTD_CLEVEL` (or equivalent if using xz)
> > > during make modules_install **depending** on the components used and
> > > their version.
> > >
> > > HTH
> > > Emil
> >
> > Hi Emil,
> >
> > Thanks for the explanations and infos.
> >
> > Sorry, for not being consistent with avoiding "top-posting".
> >
> > I took my LATEST selfmade 6.10.1 LLVM/Clang kernel and re-compressed via:
> >
> > root# zstd -T0 -19 -v *.ko
> >
> > [ SIZES ]
> >
> > 576 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/net/mac80211/mac80211.ko.zst
> > 752 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/net/mac80211/mac80211.ko.zst.orig
> >
> > 244 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst
> > 316 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst.orig
> >
> > 144 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/drivers/net/wireless/intel/iwlwifi/dvm/iwldvm.ko.zst
> > 184 /lib/modules/6.10.1-1-amd64-clang18-kcfi/kernel/drivers/net/wireless/intel/iwlwifi/dvm/iwldvm.ko.zst.orig
> >
>
> If you use the `ZSTD_CLEVEL=19 make modules_install ...` (somewhere in
> your Debian recipe) as mention earlier the extra re-compression should
> not be needed.
>
Hi Emil,
Is ZSTD_CLEVEL an Arch Linux specific variable?
git grep shows the usage of ... in the Linux-kernel source:
include/linux/zstd_lib.h:# define ZSTD_CLEVEL_DEFAULT 3
I tried Linux v6.10.3 with my "[PATCH] kbuild: Use better
compress-level for zstd compressed modules"
Result: 25MiB smaller linux-image Debian package and same size
reduction in /lib/modules/.
Link: https://lore.kernel.org/all/CA+icZUUQadYjAXiCNx7PmKDV20WctvnzkXC3R2F7FM4Gzcm39Q@xxxxxxxxxxxxxx/
> > [ INSTRUCTIONS ]
> >
> > I stopped NetworkManager.service + networking.service + removed iwldvm
> > kernel-module (modprobe -r -v iwldvm).
> >
> > Compressed 3 kernel-modules (see above).
> >
> > Reloaded modified iwldvm (modprobe -v iwldvm) + started
> > networking.service + NetworkManager.service
> >
> > WIFI OK - Logs (dmesg) OK.
> >
> > So user-space works with kernel-space here on Debian/unstable AMD64.
> >
>
> Nice, so it's working as expected.
>
> Using level 19 level is safe assuming you don't use busybox initrd,
> with zstd compressed modules. Although you can use even higher level,
> if using kmod before commit 09c9f8c5df0475d2627d9498e26250592dbfebd6 -
> kmod v30 it seems.
>
> Since I'm not everyone is a Debian person, it'll be great to
> explicitly list the versions used ... for the future.
>
> If you have particular question/concern, let me know.
>
I can NOT say if some new KConfigs make sense here to check for
pre-requisites of specific versions of kmod/zstd/etc.
Introducing CONFIG_MODULE_COMPRESS_ZSTD_CLEVEL?
Thanks for your feedback.
-Sedat-
