Thanks a lot for this Michel! Ben Cotton <bcotton@xxxxxxxxxx> writes: > https://fedoraproject.org/wiki/Changes/MemoryConstraintsMacros > > == Summary == > Introduce macros, similar to openSUSE's > [https://build.opensuse.org/package/show/openSUSE:Factory/memory-constraints > memory-constraints]), for optionally limiting build parallelism for > build-time memory-bound packages > > == Owner == > * Name: [[User:salimma|Michel Alexandre Salim]] > * Email: michel AT michel-slm DOT name > > > == Detailed Description == > Some source packages have a memory usage per build thread higher than > the RAM:CPU ratio available in some of our builders. Further, this > ratio can be different for different build server on different > architectures. > > At the moment, such packages > ([https://src.fedoraproject.org/rpms/ceph/blob/d7454e4e0a98208dc569553b901a49733beff6b3/f/ceph.spec#_1269-1390 > ceph], [https://src.fedoraproject.org/rpms/chromium/blob/baaf27b384295d6288ef367dd108ce9874543f2d/f/chromium.spec#_3-14 > chromium], [https://src.fedoraproject.org/rpms/mcrouter/blob/a0f7ecad2ccc51c4214646b082358745c7882c86/f/mcrouter.spec#_68-82 > mcrouter]) have to implement their own logic for determining the safe > amount of parallelism, and redefine `_smp_build_ncpus`. > > When this proposal is implemented, they can instead declaratively > specify the amount of RAM needed per build thread, e.g. > > %limit_build -m 8192 > > for declaring a build thread should be allocated 8GB of RAM. > > Since Koji supports > [https://docs.pagure.org/koji/release_notes/release_notes_1.18/#system-changes > setting default values for macros], there will be a macro for the > default memory limit (name TBD) that, if set, will be used to cap > `_smp_build_ncpus` unless overridden by `%limit_build -m`. > 0 > I'm proposing to tentatively call the macro package > `build-constraints-rpm-macros` to allow the possibility of adding > macros for related needs e.g. [https://pagure.io/copr/copr/issue/1678 > build timeouts] to the same package. > > > == Benefit to Fedora == > This change simplifies maintaining specs for software that are > memory-bounded rather than CPU-bounded on our build servers > > It could potentially improve build reliability for these packages, by > reducing the number of jobs failing because of OOM errors, and reduce > the need for package maintainers to debug these failures. > > By keeping the user-facing API aligned with what openSUSE is using, we > open up the possibility to collaborate with them and with the upstream > RPM project to get such macros upstreamed into RPM itself (see > [https://github.com/rpm-software-management/rpm/pull/821 previous > attempt]). **note** that is not in scope for this Change. > > == Scope == > * Proposal owners: > ** Introduce new macros > ** Update known packages to use the new macros, replacing their custom > `_smp_build_ncpus` overrides > > * Other developers: > ** The proposal owners might not catch all references of such logic. > Individual package maintainers can try refactoring their packages > using these new macros > > * Release engineering: [https://pagure.io/releng/issue/10188 #10188] > No mass rebuild needed. Affected packages should be rebuilt using the new macro > > * Policies and guidelines: Packaging guideline can be updated to > recommend using these macros for build-time memory-bound packages > > * Trademark approval: N/A (not needed for this Change) > * Alignment with Objectives: N/A > > > == Upgrade/compatibility impact == > No impact, affects package building only. Also, the use of the new > macros are optional. > > > > == How To Test == > 1. Install `build-constraints-rpm-macros` > 2. Modify spec to set `%limit_build -n AMOUNT_IN_MB` in `%build` > 3. Rebuild in koji and make sure it passes on all supported architectures > > > > == User Experience == > > > == Dependencies == > This can optionally be added as dependencies of `redhat-rpm-config` > and `epel-rpm-macros`, depending on how many packages need this > > > > == Contingency Plan == > > * Contingency mechanism: (What to do? Who will do it?) > Revert changed packages to their previous way of capping the number of > build jobs > > * Contingency deadline: beta > * Blocks release? No > > > == Documentation == > Previous discussion: > https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/thread/GWSXTGOCMNXHPGMTW32LI5EKPGHDKCFU/#GWSXTGOCMNXHPGMTW32LI5EKPGHDKCFU > > openSUSE implementation: > https://build.opensuse.org/package/show/openSUSE:Factory/memory-constraints > > > -- > Ben Cotton > He / Him / His > Fedora Program Manager > Red Hat > TZ=America/Indiana/Indianapolis > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
