|
I need help to optimize the total build time of Ceph package generation. Currently, the total average time to generate Ceph packages is 2hr+, which generates all the packages. To optimize build time, I need to generate only the required packages: https://raw.githubusercontent.com/Seagate/cortx-re/rocky-linux-8.4/docker/cortx-deploy/cortx-rgw/ceph-componenet-rpms.txt.
The ceph.spec file provides some default conditional parameters which is being used with the rpmbuild command with multicore cpu utilization as follows for optimization:
# rpmbuild --clean --rmsource --define "_unpackaged_files_terminate_build 0" --define "debug_package %{nil}" --define "_binary_payload w2T16.xzdio" --define "_topdir `pwd`" --without seastar --without cephfs_java --without ceph_test_package --without selinux
--without lttng --without cephfs_shell --without amqp_endpoint --without kafka_endpoint --without lua_packages --without zbd --without cmake_verbose_logging --without rbd_rwl_cache --without rbd_ssd_cache --without system_pmdk --without jaeger --without
ocf --without make_check -ba /mnt/rgw/SPECS/ceph.spec
To optimize build time further I explored some processes as follows:
1. Tried editing the spec file with various inbuilt parameters, specifically "%file" provides with the option to control the final rpm binary generated. By removing "%file" section for unrequired packages I can generate only the required packages rpm binary.
But this doesn’t provide with any significant boost in time as total build process is still 100+ mins. Current conclusion is that "rpmbuild" doesn’t provide much option to reduce build time but only generation of final rpm binary. To reduce build time, may
need to take a deeper look at the %build section. Reference: https://stackoverflow.com/a/22577681
2. Tried with editing the Cmake build command in the "%build" section of spec file, which provides extensive debug options to control build process. The "cmake -LAH" command lists all options provided to include/exclude various components and libraries.
The build process errored out many times because it requires deep understanding of components and libraries to exclude from Cmake.
I discovered that a feature request to exclude ceph-test sub-package from build process was raised and merged earlier: Feature #16776: https://tracker.ceph.com/issues/16776 · Pull Request #10872: https://github.com/ceph/ceph/pull/10872/files.
My current undrestanding is to implement akin to something to above pr in cortx-rgw repo to build only required packages. This would require editing the various Cmake files and spec file which also require knowledge on component level.
3. Also tried to edit the Cmake file and spec file in source tar file then build from it as mentioned above. By removing section of unrequired packages in Cmake files I see that the build process skips them during execution. But here also the build errors
out due to many linked libraries and components being tightly coupled into other components.
Currently, as per my understanding, to optimize build time I need to edit the various Cmake files and provide with options to build specific packages and then pass those option in the spec file in "%build" or "%bcond" as parameters to generate only required
packages. Is this the right approach I am headed or something else need to be done here.
Thanks,
Nitish
Seagate Internal |
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
