Re: rpmbuild is very slow with large files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 7/12/22 11:02, Marius Schwarz wrote:
> Am 12.07.22 um 10:55 schrieb Marius Schwarz:
>>
>> The rpmbuild process for this one rpm was single thread. With a
>> lsof-loop,  I could see "bytes" getting attached to the resulting file
>> with an awful slow progression rate. Which is very frustrating to see
>> on a 8 core system.
>>
>> The thing is, I do testbuilds of VOSK with language model and code
>> etc. on one of my servers. If this project ever reaches the Fedora
>> build farm,
>> we can expect a very long build time, if nothing is changed in
>> rpmbuild. Is there maybe a hidden parallel compression option somewhere?
>>
> 
> Looks like someone else had the same problem with rpmbuilds XZ single
> core compression:
> 
> https://insujang.github.io/2020-11-07/accelerating-ceph-rpm-packaging-using-multithreaded-compression/
> 
> |--define "_binary_payload w2T16.xzdio" |
> 
> |Question: Is the resulting compression format suitable for Fedora repo
> or a against a policy?|

Yeah, we did quite some work upstream to get builds run in parallel at
various stages and levels.

RPM does support multithreaded compression where the compression
libraries support that but it needs to be enabled. As the result are not
the same as single threaded compression this may have impact on the
viability of deltarpms. But IIRC at least zstd while having different
results would at least have reproducible results. But I am not the one
that actually checked and decided against threaded compression.

Florian

> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux