perhaps you should look at how ceph has dealt with a similar issue, they set the max number of cpus based on the system ram. https://src.fedoraproject.org/rpms/ceph/blob/rawhide/f/ceph.spec#_1246 Dennis On Fri, Mar 26, 2021 at 7:49 PM Michel Alexandre Salim <michel@xxxxxxxxxxxxxxx> wrote: > > Hi all, > > This idea came about when I'm debugging build issues with mcrouter, > which turns out to be due to build jobs failing to allocate memory and > getting terminated without aborting the entire compilation, causing > link issues when empty or corrupted objects are encountered: > > https://src.fedoraproject.org/rpms/mcrouter/blob/rawhide/f/mcrouter.spec#_4-8 > > As a rough estimate it seems like each of the CPU core passed with > %{_smp_build_ncpus} ended up consuming close to 8 GB of RAM. And that's > with LTO disabled (yeah, it's not a good situation to be in). > > Right now I'm just overriding _smp_build_ncpus to 1, but there is a > more elegant solution I'd like to propose: > > What if one can declaratively set the required RAM per build job -- > either with a single macro, or maybe two if the LTO usecase requires > even more RAM. e.g. to declare each core might take up to 8 GB: > > %global _smp_build_ram_per_cpu 8192 > > then in case this is run on our aarch64 builder with 40GB RAM, > dynamically take the minimum of the existing _smp_build_ncpus (which > AIUI is determined by the number of cores on the machine) and (amount > of RAM / _smp_build_ram_per_cpu), in this case capping the actual > number passed to -j to 5. > > Is there interest in having this be available? I could imagine it might > be useful for other resource-intensive package builds e.g. for > Chromium. > > Best regards, > > -- > Michel Alexandre Salim > profile: https://keyoxide.org/michel@xxxxxxxxxxxxxxx > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
