|
Hi,
I need help to optimize the total build time of Ceph package generation. Currently, the total average time to generate Ceph packages is 2hr+, which generates all the packages. To optimize build time, I need to generate only the required packages: https://raw.githubusercontent.com/Seagate/cortx-re/rocky-linux-8.4/docker/cortx-deploy/cortx-rgw/ceph-componenet-rpms.txt. The ceph.spec file provides some default conditional parameters which is being used with the rpmbuild command with multicore cpu utilization as follows for optimization: # rpmbuild --clean --rmsource --define "_unpackaged_files_terminate_build 0" --define "debug_package %{nil}" --define "_binary_payload w2T16.xzdio" --define "_topdir `pwd`" --without seastar --without
cephfs_java --without ceph_test_package --without selinux --without lttng --without cephfs_shell --without amqp_endpoint --without kafka_endpoint --without lua_packages --without zbd --without cmake_verbose_logging --without rbd_rwl_cache --without rbd_ssd_cache
--without system_pmdk --without jaeger --without ocf --without make_check -ba /mnt/rgw/SPECS/ceph.spec
To optimize build time further I explored some processes as follows:
1. Tried editing the spec file with various inbuilt parameters, specifically %file provides with the option to control the final rpm binary generated. By removing %file section for unrequired packages
we can generate only the required packages rpm binary. But this doesn’t provide with any significant boost in time as total build process is still 100+ mins. Current conclusion is that rpmbuild doesn’t provide much option to reduce build time but only generation
of final rpm binary. To reduce build time, may need to take a deeper look at the %build section. Reference: https://stackoverflow.com/a/22577681
2. Tried with editing the cmake build command in the %build section of spec file, which provides extensive debug options to control build process. The cmake -LAH command lists all options provided
to include/exclude various components and libraries. The build process errored out many times because it requires deep understanding of components and libraries to exclude from cmake.
We discovered that a feature request to exclude ceph-test subpackage from build process was raised and merged earlier: Feature #16776: https://tracker.ceph.com/issues/16776 ·
Pull Request #10872: https://github.com/ceph/ceph/pull/10872/files.
My current undrestandig is to implement akin to something to above pr in cortx-rgw repo to build only required packages. This would require editing the various cmake files and spec file which also
require knowledge on component level.
3. Also tried to edit the cmake file and spec file in source tar file then build from it as mentioned above. By removing section of unrequired packages in cmake files we see that the build process
skips them during execution. But here also the build errors out due to many linked libraries and components being tightly coupled into other components.
Thanks, Nitish
Seagate Internal |
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
