Re: Sphinx parallel build errors

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, Oct 27, 2019 at 04:05:07PM -0300, Mauro Carvalho Chehab wrote:
> Em Sun, 27 Oct 2019 11:30:22 -0700
> Randy Dunlap <rdunlap@xxxxxxxxxxxxx> escreveu:
> 
> > On 10/27/19 11:17 AM, Jonathan Corbet wrote:
> > > On Sun, 27 Oct 2019 11:11:57 -0700
> > > Randy Dunlap <rdunlap@xxxxxxxxxxxxx> wrote:
> > >   
> > >>> Yes, I see oom-kills when building docs for 5.4-rc2.
> > >>> But only after I ran 'gimp *.jpg' for 22 photos.
> > >>>
> > >>> I can see anywhere from 1 to 4 sphinx-build processes running.    
> > >>
> > >> I continue to have oom-kills when building docs.
> 
> My 2 cents here.
> 
> I would keep the default to use more CPUs, as nowadays I suspect that
> most devs have a way more than 8GB installed. The worse machine I use
> here to build Kernel is a Gen-3 i5 that has has 16GB (and it is a 
> personal laptop I bought 5 years ago).
> 
> Yet, I agree that doc build should be saner on low-cap machines.
> 
> Ideally, the fix should be, IMHO, at sphinx-build: -jauto should
> detect OOM possible conditions and reduce the number of used CPUs
> it the max available memory is too low.

How can we determine that dynamically, though? e.g. how do we determine
how much is needed per thread in a way that will move with the future
needs of Sphinx, etc?

> As there's now a script at Kernel, maybe the script could be modified 
> to detect if the max available memory is too low, passing -j1 if,
> let's say, the machine has less than (let's say) 16 GB RAM.

Randy, are you able to just use -j2 or similar with your builds to avoid
this?

-- 
Kees Cook



[Index of Archives]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]     [Linux Resources]

  Powered by Linux