[OS-BUILD PATCH] redhat/kernel.spec.template: Parallelize compression

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



From: Prarit Bhargava <prarit@xxxxxxxxxx>

redhat/kernel.spec.template: Parallelize compression

This line in the kernel.spec file:

find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz;

implies that ${RPM_BUILD_NCPUS} xz instances are run.  This is not true,
and this line actually applies ${RPM_BUILD_NCPUS} to a single instance of
xz.  This means that the compression has been done one module at a time
(ie, in serial) rather than in parallel as is implied by the code.

Rewrite this code into a script and make the execution truly parallel.

Signed-off-by: Prarit Bhargava <prarit@xxxxxxxxxx>

diff --git a/redhat/Makefile b/redhat/Makefile
index blahblah..blahblah 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -632,6 +632,7 @@ sources-rh: $(TARBALL) generate-testpatch-tmp setup-source dist-configs-check
 		scripts/mod/mod-internal.list \
 		scripts/mod/mod-partner.list \
 		scripts/mod/mod-sign.sh \
+		scripts/mod/mod-compress.sh \
 		configs/flavors \
 		configs/generate_all_configs.sh \
 		configs/merge.pl \
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -767,6 +767,8 @@ Source21: mod-sign.sh
 
 %define modsign_cmd %{SOURCE21}
 
+Source22: mod-compress.sh
+
 %if 0%{?include_rhel}
 Source23: x509.genkey.rhel
 
@@ -2472,7 +2474,8 @@ find Documentation -type d | xargs chmod u+w
     fi \
   fi \
   if [ "%{zipmodules}" -eq "1" ]; then \
-    find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz; \
+    echo "Compressing modules ..." \
+    %{SOURCE22} "$RPM_BUILD_ROOT/lib/modules/" "${RPM_BUILD_NCPUS}" \
   fi \
 %{nil}
 
diff --git a/redhat/scripts/mod/mod-compress.sh b/redhat/scripts/mod/mod-compress.sh
new file mode 100755
index blahblah..blahblah 100755
--- /dev/null
+++ b/redhat/scripts/mod/mod-compress.sh
@@ -0,0 +1,22 @@
+#!/usr/bin/bash
+
+BUILD_DIR=$1
+NCPUS=$2
+
+if [ ! -e "$BUILD_DIR" ]; then
+	echo "Could not find $BUILD_DIR"
+	exit 1
+fi
+
+modcount=0
+for module in $(find "$BUILD_DIR" -name *.ko)
+do
+	xz $module &
+	waitpids[${modcount}]=$!
+	modcount=$(expr $modcount + 1)
+	while [ "$(jobs | grep -c Running)" -ge "$NCPUS" ]; do :; done
+done
+
+for pid in ${waitpids[*]}; do
+	wait ${pid}
+done

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2254
_______________________________________________
kernel mailing list -- kernel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to kernel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/kernel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Index of Archives]     [Fedora General Discussion]     [Older Fedora Users Archive]     [Fedora Advisory Board]     [Fedora Security]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Mentors]     [Fedora Package Announce]     [Fedora Package Review]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Coolkey]     [Yum Users]     [Tux]     [Yosemite News]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [USB]     [Asterisk PBX]

  Powered by Linux