Re: RFC: declaring estimated per-builder RAM usage in spec file

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Michel,

Michel Alexandre Salim <michel@xxxxxxxxxxxxxxx> writes:

> Hi all,
>
> This idea came about when I'm debugging build issues with mcrouter,
> which turns out to be due to build jobs failing to allocate memory and
> getting terminated without aborting the entire compilation, causing
> link issues when empty or corrupted objects are encountered:
>
> https://src.fedoraproject.org/rpms/mcrouter/blob/rawhide/f/mcrouter.spec#_4-8
>
> As a rough estimate it seems like each of the CPU core passed with
> %{_smp_build_ncpus} ended up consuming close to 8 GB of RAM. And that's
> with LTO disabled (yeah, it's not a good situation to be in).
>
> Right now I'm just overriding _smp_build_ncpus to 1, but there is a
> more elegant solution I'd like to propose:
>
> What if one can declaratively set the required RAM per build job --
> either with a single macro, or maybe two if the LTO usecase requires
> even more RAM. e.g. to declare each core might take up to 8 GB:
>
> %global _smp_build_ram_per_cpu 8192
>
> then in case this is run on our aarch64 builder with 40GB RAM,
> dynamically take the minimum of the existing _smp_build_ncpus (which
> AIUI is determined by the number of cores on the machine) and (amount
> of RAM / _smp_build_ram_per_cpu), in this case capping the actual
> number passed to -j to 5.
>
> Is there interest in having this be available? I could imagine it might
> be useful for other resource-intensive package builds e.g. for
> Chromium.

openSUSE has a package and a set of macros for exactly this purpose:
https://build.opensuse.org/package/show/network:chromium/memory-constraints
(and funnily enough, it has been created for Chromium).

It is pretty simple to use, just plop this line into your spec file:
%limit_build -m MAX_MB_PER_THREAD

and the macro will set %_smp_mflags for you.


Cheers,

Dan

Attachment: signature.asc
Description: PGP signature

_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux