Re: parallel compilation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I finally understood my parallel compilation problem. 
I was not related to gnuCobol, but to an unneeded and too agressive  rm /tmp/cob* command in my compilation shell. 

Sorry for that.

Erwan



2016-09-21 22:58 GMT+02:00 Erwan Duroselle <erwan.duroselle@xxxxxxxxx>:
Hi,

I have done some more testing, and things may not be so simple:

- my parallel compilation problems are not solved by using $TMPDIR, at least when the parallel degree is higher than 2
- I cannot reproduce the error messages when using a simplified compilation shell,even if parallel degree is higher than 2.

So the problem may be related to my compilation shell, which is rather complex, not to gnuCobol.
I'll try to write a reproductible test case, with a simple compilation shell, and let you know if it seems to be related to gnucobol or not.

Sorry if I raised a false alert.

BTW: I like the idea of tmpfs. I'll test it one day.

Erwan

2016-09-19 20:45 GMT+02:00 Simon Sobisch <simonsobisch@xxxxxx>:

This is really strange. cobc uses
cob_temp_name(), see cobc.c around line 2901

This is defined in common.c around line 3281 and uses:

    cob_gettmpdir(), SLASH_CHAR, cob_sys_getpid(), cob_temp_iteration, ext
 
You can override the temporary direcory which you did (first part), but this shouldn't be necessary as the second part is the process-id which shouldn't be reused within a time frame needed for compilation.

This is quite strange and should be investigated further!

BTW: if you want to do parallel builds I suggest not to do this manually but use a small makefile with wildcard, this allows you to do

    make -j16 all # generate all sources that are older than the binaries with 16 parallel jobs (depending on the number of logical cores your cpu has and if you or other people want to do other stuff on the server in the meanwhile)

BTW#2: if you do a mass and parallel compilation I'd suggest to have TMPDIR (not globally but for cobc, for example in the makefile) set to a tmpfs or ramfs mount, this can speed up the compilation measurably

Simon

> Problem solved with $TMPDIR,
> Thanks.
>
> Erwan
>
> 2016-09-19 10:05 GMT+02:00 Boepple, Florian <florian.boepple@xxxxxxxxx>:
> Hi Erwan,
>
> you can set the environment variable $TMPDIR.
> If this is not set GnuCOBOL will look in $TMP and also $TEMP for a value.
> If none of these are set "/tmp" will be used.
>
> See function cob_gettmpdir() in libcob/common.c
>
> Best regards
> Florian
>
>
>
> Von: Erwan Duroselle [mailto:erwan.duroselle@gmail.com]
> Gesendet: Montag, 19. September 2016 09:16
> An: open-cobol-list@lists.sourceforge.net
> Betreff: parallel compilation
>
> Hi,
>
> I'm a rather new to gnuCobol, so please forgive any obvious question.
>
> I am trying to compile a rather large software (many thousands of
> programs), issued from AS400 and Microfocus, with gnuCobol 2.0 rc1, on a
> Centos VM.
>
> Most of my programs compile wonderfully, though I may ask questions about
> the few remaining ones later.
>
> My problem is when I try to compile several programs in parallel, to save
> compilation time.
> Say A* programs runs in a shell, and B* in an other one, at the same time.
> On about 5% of the programs, apparently randomly, I get errors like:
>
> /tmp/cob6501_0.c:24:30: error: /tmp/cob6501_0.c.h : No such file or
> directory
> /tmp/cob6501_0.c:47:34: error: /tmp/cob6501_0.c.l.h : No such file or
> directory
> /tmp/cob6501_0.c: In function ?BTRX030C_?:
> /tmp/cob6501_0.c:59: error: ?module? undeclared (first use in this
> function)
> /tmp/cob6501_0.c:59: error: (Each undeclared identifier is reported only
> once
> /tmp/cob6501_0.c:59: error: for each function it appears in.)
> /tmp/cob6501_0.c:59: error: ?cob_glob_ptr? undeclared (first use in this
> function)
> /tmp/cob6501_0.c:62: error: ?cob_procedure_params? undeclared (first use
> in this function)
> /tmp/cob6501_0.c:65: error: ?frame_ptr? undeclared (first use in this
> function)
> /tmp/cob6501_0.c:65: error: ?frame_stack? undeclared (first use in this
> function)
>
> When compiled alone, those programs compile without any message.
> When parallel compilation is launched again, simillar errors will happen
> on other programs, not on the same ones.
> The same problem happens on an older 2.0 version from 2015, on an other
> RedHat box.
>
> Does anyone have a clue on how to avoid these errors?
> I have tried to set up a temporary compilation directory, different from
> /tmp, but couldn't find a parameter for that.
>
> Regards,
> Erwan
>


------------------------------------------------------------------------------
_______________________________________________
open-cobol-list mailing list
open-cobol-list@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/open-cobol-list

[Index of Archives]     [Gcc Help]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Info]     [Linux Kernel]     [Linux SCSI]     [Big List of Linux Books]

  Powered by Linux