On Wed, 4 Oct 2023 at 09:48, Cristian Le via devel <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
Thank you Stephen,
Didn't know we can have such control over koji scheduling. Is there any documentation on koji side or fedora packaging guidelines about this? E.g. what are the physical limits that can be queued, how to request specific number of CPUs for mpi packages etc.? Which macro sets take this into account? I believe %ctest ones do that, but I've found an MPI related situation where I need to override this.
Uhm sorry, I am communicating poorly today it would seem. The packager does not have any control on the number of CPUs, the amount of memory, amount of disk space, etc. Those are set by infrastructure in how builders are installed as virtual machines. Infrastructure in turn has to deal with hard limits of how much disk space, cpu, memory are in the systems infrastructure have which is further hard limited by the amount of rack space, power, and budgets that they get handed to them. Most of these are hard choices made by releng infrastructure on how to best divy up the resources to deal with lots of small builds and regular large builds.
What a packager can control is how the package is configured and what options are given to subparts of the 'build' inside of the RPM: config options, make options, and debug size options. A packager could choose to use fewer CPUs by configuring their build to use 1 CPU (make -j1 ) or possibly less memory, but it would need to be options set in whatever equivalent to Makefiles etc that are there.
I hope that better explains things.. if it doesn't... I ask anyone else to better explain things.
_______________________________________________Also are the options equivalent in copr as well?
On 2023/10/04 15:32, Stephen Smoogen wrote:
On Wed, 4 Oct 2023 at 09:24, Cristian Le via devel <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
Hi Stephen,
Could you also share the links for where the strategies for large builds is, I am also curious about this? Is it within koji or another framework?
Sorry I meant the email Kalev Lember sent out
# Require 32 GB of RAM per CPU core for debuginfo processing
%global _find_debuginfo_opts %limit_build -m 32768
... and see if it helps. With this, it should pass -j1 to find-debuginfo when the builder has 32 GB RAM, -j2 when it has 64 GB etc.
Other strategies are to limit the number of parallel parts of a build (aka make or cmake flags) happening at the same time
Outside of that it is mainly up to a package and what it uses. The problem is that there are limited resources (number of builders, amount of memory per builder, amount of cpu) and as a packager you have to manage thatOn 2023/10/04 14:41, Stephen Smoogen wrote:
On Wed, 4 Oct 2023 at 05:44, Martin Stransky <stransky@xxxxxxxxxx> wrote:
Hello guys,
Is there's a way how to set requested amount of ram for koji builders?
No, the builders are mainly virtual machines on a set of hardware with a set amount of memory per virtual machine. There are some 'real' hardware servers but it is mainly a round robin to get them. As noted elsewhere there are other strategies to use with large builds.
--
Stephen Smoogen, Red Hat AutomotiveLet us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Stephen Smoogen, Red Hat Automotive
Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
