Re: kbuild: add support for zstd compressed modules

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Apr 8, 2021 at 9:11 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
>
> On Thu, Apr 8, 2021 at 9:04 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote:
> >
> > Hi,
> >
> > I tried this patch together with my build-script which uses builddeb
> > to generate Debian packages.
> >
> > Currently, I test a setup with Clang-LTO + DWARF-v5 (and BTF).
> >
> > With this patch I get total strange numbers.
> >
> > The Kconfig changes to my previous kernel of today:
> >
> > $ scripts/diffconfig /boot/config-5.12.0-rc6-3-amd64-clang12-lto
> > /boot/config-5.12.0-rc6-4-amd64-clang12-lto
> > BUILD_SALT "5.12.0-rc6-3-amd64-clang12-lto" -> "5.12.0-rc6-4-amd64-clang12-lto"
> > MODULE_COMPRESS_NONE y -> n
> > MODULE_COMPRESS_ZSTD n -> y
> >
> > The generated linux-image Debian package is greater in size than the
> > linux-image DEBUG package.
> > Normally, this is vice-versa.
> >
> > With:
> >
> > $ du -s -m linux-image-5.12.0-rc6-4-amd64-clang12-lto*
> > 969     linux-image-5.12.0-rc6-4-amd64-clang12-lto_5.12.0~rc6-4~bullseye+dileks1_amd64.deb
> > 97      linux-image-5.12.0-rc6-4-amd64-clang12-lto-dbg_5.12.0~rc6-4~bullseye+dileks1_amd64.deb
> > (DEBUG)
> >
> > Without:
> >
> > $ du -s -m linux-image-5.12.0-rc6-3-amd64-clang12-lto*
> > 60      linux-image-5.12.0-rc6-3-amd64-clang12-lto_5.12.0~rc6-3~bullseye+dileks1_amd64.deb
> > 599     linux-image-5.12.0-rc6-3-amd64-clang12-lto-dbg_5.12.0~rc6-3~bullseye+dileks1_amd64.deb
> > (DEBUG)
> >
> > The kernel-modules directory numbers:
> >
> > With:
> >
> > # du -s -m /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/
> > 999     /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/
> >
> > Without:
> >
> > # du -s -m /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/
> > 354     /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/
> >
> > Checking iwlwifi kernel-module:
> >
> > With:
> >
> > # find /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/ -name
> > 'iwlwifi.ko*' -type f -print | xargs ls -l
> > -rw-r--r-- 1 root root 2381066  8. Apr 15:47
> > /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst
> >
> > # find /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/ -name
> > 'iwlwifi.ko*' -type f -print | xargs du -k
> > 2328    /lib/modules/5.12.0-rc6-4-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst
> >
> > Without:
> >
> > # find /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/ -name
> > 'iwlwifi.ko*' -type f -print | xargs ls -l
> > -rw-r--r-- 1 root root 675976  8. Apr 09:35
> > /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
> >
> > # find /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/ -name
> > 'iwlwifi.ko*' -type f -print | xargs du -k
> > 664     /lib/modules/5.12.0-rc6-3-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
> >
> > My make invocation looks like this
> > (start-build_5.12.0-rc6-4-amd64-clang12-lto.txt with this patch):
> >
> > dileks     74706   74681  0 15:46 pts/1    00:00:00 /usr/bin/perf_5.10
> > stat make V=1 -j4 LLVM=1 LLVM_IAS=1 PAHOLE=/opt/pahole/bin/pahole
> > LOCALVERSION=-4-amd64-clang12-lto KBUILD_BUILD_HOST=iniza
> > KBUILD_BUILD_USER=sedat.dilek@xxxxxxxxx
> > KBUILD_BUILD_TIMESTAMP=2021-04-08 bindeb-pkg
> > KDEB_PKGVERSION=5.12.0~rc6-4~bullseye+dileks1
> >
> > Finally:
> >
> > I can *NOT* boot into the kernel with this patch applied.
> >
> > Personally, I do disable intentionally:
> >
> > CONFIG_DEBUG_INFO_COMPRESSED=n
> >
> > ...and will do independent of this patch:
> >
> > CONFIG_MODULE_COMPRESS_NONE=n
> >
> > As all Debian packages - generated by builddeb - are by default
> > compressed with XZ.
> > Compressing stuff before is counterproductive.
> > This strategy generates in total smaller Debian package binaries.
> >
> > This is my first analysis, I had no deeper look.
> >
> > If you need further information, please let me know.
> >
> > Please take a look at this.
> >
> > Both linux-configs attached.
> >
> > Thanks.
> >
>
> Additional information:
>
> $ LC_ALL=C ll drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
> -rw-r--r-- 1 dileks dileks 8.2M Apr  8 19:22
> drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
>
> $ file drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
> drivers/net/wireless/intel/iwlwifi/iwlwifi.ko: ELF 64-bit LSB
> relocatable, x86-64, version 1 (SYSV),
> BuildID[sha1]=bfb188058d16a7cb7d579f68edcd1fff2a4d45d7, with
> debug_info, not stripped
>
> $ LC_ALL=C ll debian/linux-image/lib/modules/5.12.0-rc6-4-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst
> -rw-r--r-- 1 dileks dileks 2.3M Apr  8 19:29
> debian/linux-image/lib/modules/5.12.0-rc6-4-amd64-clang12-lto/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko.zst
>
> Looks like the kernel-module with debug_info gets ZTSD compressed.
>

Maybe drop `--rm` in scripts/Makefile.modinst:

+quiet_cmd_zstd = ZSTD    $@
+      cmd_zstd = $(ZSTD) -T0 --rm -f -q $<

$ zstd --help | grep '\-\-rm'
--rm    : remove source file(s) after successful de/compression

...untested.
( None of the other compressors do a removal. )

- Sedat -



[Index of Archives]     [Linux&nblp;USB Development]     [Linux Media]     [Video for Linux]     [Linux Audio Users]     [Yosemite Secrets]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux