On Tue, Mar 30, 2021 at 4:46 AM Panu Matilainen <pmatilai@xxxxxxxxxx> wrote: > > On 3/29/21 8:17 PM, Michel Alexandre Salim wrote: > > Hi again, > > > > On Fri, 2021-03-26 at 17:29 -0700, Michel Alexandre Salim wrote: > >> > > [snip] > >> Right now I'm just overriding _smp_build_ncpus to 1, but there is a > >> more elegant solution I'd like to propose: > >> > >> What if one can declaratively set the required RAM per build job -- > >> either with a single macro, or maybe two if the LTO usecase requires > >> even more RAM. e.g. to declare each core might take up to 8 GB: > >> > >> %global _smp_build_ram_per_cpu 8192 > >> > >> then in case this is run on our aarch64 builder with 40GB RAM, > >> dynamically take the minimum of the existing _smp_build_ncpus (which > >> AIUI is determined by the number of cores on the machine) and (amount > >> of RAM / _smp_build_ram_per_cpu), in this case capping the actual > >> number passed to -j to 5. > >> > >> Is there interest in having this be available? I could imagine it > >> might > >> be useful for other resource-intensive package builds e.g. for > >> Chromium. > >> > > > > Thanks to Dennis, Dan and Miroslav for the helpful pointers! > > > > Short-term I'll look into adopting something similar to the Ceph > > script, but medium-term - maybe we can get the scriptlet into redhat- > > rpm-config and then eventually have it in RPM itself once Panu's patch > > is reworked? > > > > (Having it in redhat-rpm-config or some other RPM is probably needed > > anyway, for older supported releases, so ideally we use the same macros > > and only override them in redhat-rpm-config if they were undefined). > > > > cc:ing Panu for context on the RPM PR status. > > There's no progress on that front, other than occasionally thinking > about it. > > It might not be a bad idea to be able to put an actual BuildRequire on > the amount of memory (and other similar - disk space also comes to mind) > into specs. Let's *not*. The amount of space saved in storage by putting socks and underwear in their own separate drawers os often overwhelmed by the space wasted in giving those drawers sides and a way to slide in and out. In other words, it creates unnecessary overhead in managing build environment resources. It's also difficult to predict in advance, especially for the worst offenders, such as builds whose upstream components are often pulled in dynamically: "pip install", "cpan", "mvn", "gradle", "rubygem", and my recent favorite for unpredictable and only erratically evailble dependencies, "go". While well designed SRPMs and .spec files to rely only on other RPMs for their dependencies, many in the field do not. And even when the dependencies are available, even a small change in "test" procedures can massively expand RAM and disk use during compilation. In other words, the memory requirements are likely to diverge, *massively* and unpredictably. It's safer and simpler to simply allocate memory generously for build environments. _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
