Re: Sphinx-doc: build over N processes in parallel

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Em Fri, 5 Aug 2016 11:56:44 +0200
Markus Heiser <markus.heiser@xxxxxxxxxxx> escreveu:

> Am 20.07.2016 um 16:04 schrieb Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxxx>:
> 
> > 
> > A completely unrelated question: it seems that Sphinx is using just
> > one CPU to do its builds:
> > 
> > %Cpu0  :  3,0 us,  7,6 sy,  0,0 ni, 89,4 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
> > %Cpu1  :100,0 us,  0,0 sy,  0,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
> > %Cpu2  :  1,3 us,  2,7 sy,  0,0 ni, 95,7 id,  0,3 wa,  0,0 hi,  0,0 si,  0,0 st
> > %Cpu3  :  1,0 us,  3,3 sy,  0,0 ni, 95,7 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
> > KiB Mem : 15861876 total,  5809820 free,  1750528 used,  8301528 buff/cache
> > KiB Swap:  8200188 total,  8200188 free,        0 used. 13382964 avail Mem 
> > 
> >  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     
> > 5660 mchehab   20   0  325256  89776   8300 R  99,7  0,6   0:22.25 sphinx-bui+ 
> > 
> > Are there any way to speed it up and make it use all available CPUs?  
> 
> Hi Mauro, 
> 
> sorry for the late reply. There is a sphinx-build option "-j N" [1].
> It is in a *experimental* state in Sphinx v1.2 and has been improved 
> in v1.3. Set e.g. "-j2" to the SPHINXOPTS to use two cores.
> 
>  make SPHINXOPTS=-j2 htmldocs
> 
> But take into account what the documentation says: """not all parts and 
> not all builders of Sphinx can be parallelized.""".
> 
> [1] http://www.sphinx-doc.org/en/stable/invocation.html#cmdoption-sphinx-build-j

Good, thanks!

Did some tests here on a machine with 32 CPU threads using a PCIe SSD disk,
using Sphinx 1.4.5.

Using -j32, those are the timings:

real	0m59.522s
user	1m29.968s
sys	0m4.975s

not using it, I got:

real	1m27.814s
user	1m26.465s
sys	0m1.842s

Not much gain :(

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux