Re: rpmbuild is very slow with large files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




The only coincidence is "large files" the rest is different but may
worth check , if you are talking about rawhide. 

https://pagure.io/copr/copr/issue/2241



On Tue, 2022-07-12 at 00:52 +0200, Marius Schwarz wrote:
> 
> Hi,
> 
> I have just create(d/ not finished yet, started 15 minutes ago) a
> ~2.5 
> GB rpm and found, that rpmbuild is an extrem bottleneck.
> 
> IMHO, this is caused by a fileread function which reads files in 32k 
> blocks, which is very slow and extrem IO intensive.  The result is a 
> task running at 1 core at 100% perma. With changes to larger chunks,
> we 
> can speed up so many build tasks on the farm.
> 
> Multicore use would also be helpful i.e. while packing the files.
> 
> Any counter-arguments ?
> 
> strace example:
> 
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=477601377}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=477685727}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=477892054}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478000076}) = 0
> [pid 2604060] read(5, "_I @_I s_I t_E\nauss\303\244het 
> auss\303\244h"..., 32768) = 32768
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478212651}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478301347}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478409015}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478505273}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478701366}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478784826}) = 0
> [pid 2604060] read(5, " Y_I k_I t_E\naustun austun 'aU_B"..., 32768)
> = 32768
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=478962539}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479045029}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479130924}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479213446}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479407336}) = 0
> [pid 2604060] clock_gettime(CLOCK_REAobjections
> LTIME, {tv_sec=1657579222, tv_nsec=479489832}) = 0
> [pid 2604060] read(5, "s_I v_I u:_I k_I s_I @_I s_E\naus"..., 32768)
> = 32768
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479720335}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479803090}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=479950309}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=480067186}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=480305924}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=480417985}) = 0
> [pid 2604060] read(5, "B s_I ts_I u:_I g_I R_I E_I n_I "..., 32768) =
> 32768
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=480654716}) = 0
> [pid 2604060] clock_gettime(CLOCK_REALTIME, {tv_sec=1657579222, 
> tv_nsec=480763606}) = 0
> 
> and I don't think, this tasks needs to read the clock that often too.
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam on the list, report it:
> https://pagure.io/fedora-infrastructure

-- 
Sérgio M. B.
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux