On Thu, Jun 27, 2019 at 5:33 PM Mikail Yayla
<mikail.yayla@xxxxxxxxxxxxxx> wrote:
>
> Thanks for your reply. I faced the same issues with C. How much ram do use on the machine you are compiling the forests on ?
I didn't have any current data, so I ran a new compilation on new
hardware with a relatively newer compiler (8.2.0 built myself from
source). The results were better than previous at 35 minutes compile
time. The following is for a 1.3 GB source file with 15.5 million
lines, with the first part being GCC's output and the second part
being time's output. It looks like it needed 1.5 GB memory:
$ /usr/bin/time --verbose gcc a.c -c -o delete.o -ftime-report -O3 -march=native
Time variable usr sys
wall GGC
phase setup : 0.00 ( 0%) 0.00 ( 0%)
0.00 ( 0%) 1508 kB ( 0%)
phase parsing : 94.09 ( 5%) 91.01 ( 70%)
185.02 ( 9%) 2490103 kB ( 7%)
phase opt and generate :1825.26 ( 95%) 37.61 (
29%)1862.19 ( 91%)35542183 kB ( 93%)
phase last asm : 3.22 ( 0%) 0.13 ( 0%)
3.35 ( 0%) 98002 kB ( 0%)
garbage collection : 22.95 ( 1%) 1.44 ( 1%)
24.36 ( 1%) 0 kB ( 0%)
dump files : 0.04 ( 0%) 0.08 ( 0%)
0.10 ( 0%) 0 kB ( 0%)
callgraph construction : 5.85 ( 0%) 0.26 ( 0%)
6.15 ( 0%) 786668 kB ( 2%)
callgraph optimization : 1.70 ( 0%) 0.01 ( 0%)
1.89 ( 0%) 12 kB ( 0%)
ipa function summary : 3.96 ( 0%) 0.02 ( 0%)
3.85 ( 0%) 1007 kB ( 0%)
ipa devirtualization : 0.00 ( 0%) 0.00 ( 0%)
0.01 ( 0%) 0 kB ( 0%)
ipa cp : 1.20 ( 0%) 0.00 ( 0%)
1.26 ( 0%) 205 kB ( 0%)
ipa inlining heuristics : 0.52 ( 0%) 0.00 ( 0%)
0.54 ( 0%) 0 kB ( 0%)
ipa pure const : 3.58 ( 0%) 0.07 ( 0%)
3.54 ( 0%) 1148 kB ( 0%)
ipa icf : 2.83 ( 0%) 0.12 ( 0%)
2.95 ( 0%) 0 kB ( 0%)
ipa SRA : 0.00 ( 0%) 0.00 ( 0%)
0.01 ( 0%) 0 kB ( 0%)
cfg construction : 3.28 ( 0%) 0.08 ( 0%)
3.52 ( 0%) 401229 kB ( 1%)
cfg cleanup : 60.42 ( 3%) 0.07 ( 0%)
60.63 ( 3%) 733716 kB ( 2%)
trivially dead code : 8.86 ( 0%) 0.00 ( 0%)
8.84 ( 0%) 0 kB ( 0%)
df scan insns : 6.43 ( 0%) 0.00 ( 0%)
6.32 ( 0%) 23 kB ( 0%)
df multiple defs : 5.68 ( 0%) 0.00 ( 0%)
5.45 ( 0%) 0 kB ( 0%)
df reaching defs : 15.96 ( 1%) 0.00 ( 0%)
15.92 ( 1%) 0 kB ( 0%)
df live regs : 57.93 ( 3%) 0.00 ( 0%)
57.09 ( 3%) 0 kB ( 0%)
df live&initialized regs : 33.13 ( 2%) 0.01 ( 0%)
34.50 ( 2%) 0 kB ( 0%)
df must-initialized regs : 19.95 ( 1%) 0.00 ( 0%)
20.01 ( 1%) 0 kB ( 0%)
df use-def / def-use chains : 5.77 ( 0%) 0.00 ( 0%)
6.02 ( 0%) 0 kB ( 0%)
df reg dead/unused notes : 25.88 ( 1%) 0.04 ( 0%)
25.82 ( 1%) 559983 kB ( 1%)
register information : 3.42 ( 0%) 0.00 ( 0%)
3.22 ( 0%) 0 kB ( 0%)
alias analysis : 20.58 ( 1%) 0.00 ( 0%)
21.35 ( 1%) 1280256 kB ( 3%)
alias stmt walking : 9.64 ( 1%) 5.32 ( 4%)
15.48 ( 1%) 62 kB ( 0%)
register scan : 3.05 ( 0%) 0.00 ( 0%)
2.97 ( 0%) 0 kB ( 0%)
rebuild jump labels : 7.25 ( 0%) 0.41 ( 0%)
7.95 ( 0%) 0 kB ( 0%)
preprocessing : 33.00 ( 2%) 22.51 ( 17%)
54.95 ( 3%) 204148 kB ( 1%)
lexical analysis : 25.95 ( 1%) 45.06 ( 35%)
70.75 ( 3%) 0 kB ( 0%)
parser (global) : 0.02 ( 0%) 0.01 ( 0%)
0.00 ( 0%) 1400 kB ( 0%)
parser function body : 33.31 ( 2%) 23.43 ( 18%)
57.50 ( 3%) 2284437 kB ( 6%)
inline parameters : 8.78 ( 0%) 0.09 ( 0%)
8.95 ( 0%) 1294 kB ( 0%)
integration : 0.04 ( 0%) 0.01 ( 0%)
0.05 ( 0%) 6643 kB ( 0%)
tree gimplify : 19.63 ( 1%) 1.35 ( 1%)
20.98 ( 1%) 4904633 kB ( 13%)
tree eh : 2.27 ( 0%) 0.00 ( 0%)
2.28 ( 0%) 172 kB ( 0%)
tree CFG construction : 6.70 ( 0%) 0.54 ( 0%)
7.37 ( 0%) 2432905 kB ( 6%)
tree CFG cleanup : 89.98 ( 5%) 0.22 ( 0%)
88.62 ( 4%) 1126699 kB ( 3%)
tree tail merge : 106.47 ( 6%) 0.00 ( 0%)
106.69 ( 5%) 63460 kB ( 0%)
tree VRP : 17.73 ( 1%) 0.12 ( 0%)
18.32 ( 1%) 35989 kB ( 0%)
tree Early VRP : 3.33 ( 0%) 0.00 ( 0%)
3.14 ( 0%) 4303 kB ( 0%)
tree copy propagation : 3.92 ( 0%) 0.00 ( 0%)
4.09 ( 0%) 2084 kB ( 0%)
tree PTA : 9.19 ( 0%) 0.01 ( 0%)
9.44 ( 0%) 152 kB ( 0%)
tree PHI insertion : 0.84 ( 0%) 0.03 ( 0%)
1.04 ( 0%) 256837 kB ( 1%)
tree SSA rewrite : 4.34 ( 0%) 0.27 ( 0%)
4.62 ( 0%) 868231 kB ( 2%)
tree SSA other : 4.46 ( 0%) 4.60 ( 4%)
9.18 ( 0%) 70 kB ( 0%)
tree SSA incremental : 12.42 ( 1%) 0.12 ( 0%)
12.49 ( 1%) 525620 kB ( 1%)
tree operand scan : 38.38 ( 2%) 9.45 ( 7%)
46.93 ( 2%) 837640 kB ( 2%)
dominator optimization : 264.70 ( 14%) 0.46 ( 0%)
264.95 ( 13%) 2762703 kB ( 7%)
backwards jump threading : 0.80 ( 0%) 0.01 ( 0%)
0.79 ( 0%) 176 kB ( 0%)
tree SRA : 5.77 ( 0%) 1.68 ( 1%)
7.32 ( 0%) 64297 kB ( 0%)
isolate eroneous paths : 0.74 ( 0%) 0.00 ( 0%)
0.79 ( 0%) 0 kB ( 0%)
tree CCP : 13.01 ( 1%) 0.00 ( 0%)
13.49 ( 1%) 222 kB ( 0%)
tree PHI const/copy prop : 0.32 ( 0%) 0.00 ( 0%)
0.41 ( 0%) 28 kB ( 0%)
tree split crit edges : 1.07 ( 0%) 0.03 ( 0%)
0.99 ( 0%) 292721 kB ( 1%)
tree reassociation : 2.55 ( 0%) 0.00 ( 0%)
2.25 ( 0%) 0 kB ( 0%)
tree PRE : 9.07 ( 0%) 0.24 ( 0%)
9.46 ( 0%) 704185 kB ( 2%)
tree FRE : 50.06 ( 3%) 4.95 ( 4%)
54.72 ( 3%) 1356907 kB ( 4%)
tree code sinking : 3.03 ( 0%) 0.09 ( 0%)
3.43 ( 0%) 703792 kB ( 2%)
tree linearize phis : 4.67 ( 0%) 0.63 ( 0%)
5.42 ( 0%) 351072 kB ( 1%)
tree backward propagate : 0.42 ( 0%) 0.00 ( 0%)
0.51 ( 0%) 0 kB ( 0%)
tree forward propagate : 12.18 ( 1%) 0.89 ( 1%)
12.89 ( 1%) 635818 kB ( 2%)
tree phiprop : 0.21 ( 0%) 0.01 ( 0%)
0.26 ( 0%) 0 kB ( 0%)
tree conservative DCE : 12.30 ( 1%) 0.60 ( 0%)
13.47 ( 1%) 256000 kB ( 1%)
tree aggressive DCE : 7.00 ( 0%) 0.42 ( 0%)
6.93 ( 0%) 1098 kB ( 0%)
tree buildin call DCE : 0.19 ( 0%) 0.00 ( 0%)
0.16 ( 0%) 0 kB ( 0%)
tree DSE : 65.64 ( 3%) 0.00 ( 0%)
65.67 ( 3%) 0 kB ( 0%)
PHI merge : 0.23 ( 0%) 0.00 ( 0%)
0.28 ( 0%) 19 kB ( 0%)
tree loop invariant motion : 1.87 ( 0%) 0.00 ( 0%)
1.79 ( 0%) 0 kB ( 0%)
scev constant prop : 0.10 ( 0%) 0.00 ( 0%)
0.03 ( 0%) 7 kB ( 0%)
complete unrolling : 0.02 ( 0%) 0.00 ( 0%)
0.02 ( 0%) 577 kB ( 0%)
tree slp vectorization : 1.87 ( 0%) 0.00 ( 0%)
1.76 ( 0%) 81741 kB ( 0%)
tree loop distribution : 0.21 ( 0%) 0.00 ( 0%)
0.28 ( 0%) 0 kB ( 0%)
tree iv optimization : 0.04 ( 0%) 0.00 ( 0%)
0.04 ( 0%) 0 kB ( 0%)
tree copy headers : 0.02 ( 0%) 0.01 ( 0%)
0.03 ( 0%) 645 kB ( 0%)
tree SSA uncprop : 0.64 ( 0%) 0.00 ( 0%)
0.81 ( 0%) 0 kB ( 0%)
tree switch conversion : 0.07 ( 0%) 0.00 ( 0%)
0.07 ( 0%) 0 kB ( 0%)
tree switch lowering : 0.09 ( 0%) 0.00 ( 0%)
0.06 ( 0%) 0 kB ( 0%)
gimple CSE sin/cos : 0.06 ( 0%) 0.00 ( 0%)
0.04 ( 0%) 0 kB ( 0%)
gimple widening/fma detection : 0.65 ( 0%) 0.00 ( 0%)
0.65 ( 0%) 0 kB ( 0%)
tree strlen optimization : 0.51 ( 0%) 0.00 ( 0%)
0.46 ( 0%) 0 kB ( 0%)
dominance frontiers : 4.17 ( 0%) 0.00 ( 0%)
4.85 ( 0%) 0 kB ( 0%)
dominance computation : 48.80 ( 3%) 0.38 ( 0%)
48.47 ( 2%) 0 kB ( 0%)
control dependences : 2.27 ( 0%) 0.00 ( 0%)
2.21 ( 0%) 0 kB ( 0%)
out of ssa : 3.56 ( 0%) 0.00 ( 0%)
3.61 ( 0%) 89 kB ( 0%)
expand vars : 9.21 ( 0%) 0.02 ( 0%)
9.47 ( 0%) 183298 kB ( 0%)
expand : 24.82 ( 1%) 0.43 ( 0%)
25.20 ( 1%) 3841647 kB ( 10%)
post expand cleanups : 12.22 ( 1%) 0.68 ( 1%)
12.33 ( 1%) 1882283 kB ( 5%)
lower subreg : 0.04 ( 0%) 0.00 ( 0%)
0.09 ( 0%) 0 kB ( 0%)
forward prop : 13.95 ( 1%) 0.04 ( 0%)
14.04 ( 1%) 302340 kB ( 1%)
CSE : 52.01 ( 3%) 0.01 ( 0%)
52.40 ( 3%) 5982 kB ( 0%)
dead code elimination : 10.14 ( 1%) 0.00 ( 0%)
10.15 ( 0%) 0 kB ( 0%)
dead store elim1 : 9.66 ( 1%) 0.02 ( 0%)
9.36 ( 0%) 148482 kB ( 0%)
dead store elim2 : 9.44 ( 0%) 0.01 ( 0%)
9.58 ( 0%) 137841 kB ( 0%)
loop analysis : 0.10 ( 0%) 0.00 ( 0%)
0.08 ( 0%) 0 kB ( 0%)
loop init : 17.64 ( 1%) 0.02 ( 0%)
18.14 ( 1%) 3420 kB ( 0%)
loop fini : 0.39 ( 0%) 0.00 ( 0%)
0.35 ( 0%) 0 kB ( 0%)
CPROP : 61.25 ( 3%) 0.16 ( 0%)
61.61 ( 3%) 1606298 kB ( 4%)
PRE : 43.75 ( 2%) 0.00 ( 0%)
43.98 ( 2%) 0 kB ( 0%)
CSE 2 : 42.86 ( 2%) 0.00 ( 0%)
42.99 ( 2%) 9492 kB ( 0%)
branch prediction : 5.74 ( 0%) 0.00 ( 0%)
5.73 ( 0%) 665 kB ( 0%)
combiner : 40.86 ( 2%) 0.18 ( 0%)
41.03 ( 2%) 1335301 kB ( 4%)
if-conversion : 12.47 ( 1%) 0.01 ( 0%)
12.08 ( 1%) 130958 kB ( 0%)
integrated RA : 56.03 ( 3%) 0.03 ( 0%)
55.98 ( 3%) 1817584 kB ( 5%)
LRA non-specific : 27.22 ( 1%) 0.02 ( 0%)
27.14 ( 1%) 27 kB ( 0%)
LRA virtuals elimination : 1.97 ( 0%) 0.00 ( 0%)
2.12 ( 0%) 187 kB ( 0%)
LRA create live ranges : 7.22 ( 0%) 0.00 ( 0%)
7.04 ( 0%) 0 kB ( 0%)
LRA hard reg assignment : 1.88 ( 0%) 0.00 ( 0%)
2.03 ( 0%) 0 kB ( 0%)
reload : 0.23 ( 0%) 0.00 ( 0%)
0.18 ( 0%) 0 kB ( 0%)
reload CSE regs : 27.27 ( 1%) 0.11 ( 0%)
27.23 ( 1%) 458199 kB ( 1%)
load CSE after reload : 4.93 ( 0%) 0.00 ( 0%)
4.50 ( 0%) 0 kB ( 0%)
ree : 6.81 ( 0%) 0.00 ( 0%)
6.84 ( 0%) 56956 kB ( 0%)
thread pro- & epilogue : 0.93 ( 0%) 0.00 ( 0%)
0.79 ( 0%) 442 kB ( 0%)
if-conversion 2 : 1.58 ( 0%) 0.00 ( 0%)
1.65 ( 0%) 40 kB ( 0%)
split paths : 0.27 ( 0%) 0.00 ( 0%)
0.27 ( 0%) 0 kB ( 0%)
combine stack adjustments : 1.12 ( 0%) 0.00 ( 0%)
1.14 ( 0%) 0 kB ( 0%)
peephole 2 : 7.39 ( 0%) 0.11 ( 0%)
7.56 ( 0%) 450261 kB ( 1%)
hard reg cprop : 8.19 ( 0%) 0.00 ( 0%)
8.12 ( 0%) 11400 kB ( 0%)
scheduling 2 : 66.85 ( 3%) 0.02 ( 0%)
67.00 ( 3%) 154655 kB ( 0%)
machine dep reorg : 8.48 ( 0%) 0.00 ( 0%)
8.45 ( 0%) 484 kB ( 0%)
reorder blocks : 6.98 ( 0%) 0.00 ( 0%)
6.19 ( 0%) 103733 kB ( 0%)
shorten branches : 7.98 ( 0%) 0.00 ( 0%)
8.01 ( 0%) 0 kB ( 0%)
final : 14.32 ( 1%) 0.43 ( 0%)
14.88 ( 1%) 522494 kB ( 1%)
tree if-combine : 0.34 ( 0%) 0.00 ( 0%)
0.32 ( 0%) 0 kB ( 0%)
straight-line strength reduction : 0.65 ( 0%) 0.00 ( 0%)
0.66 ( 0%) 70 kB ( 0%)
store merging : 0.14 ( 0%) 0.00 ( 0%)
0.07 ( 0%) 72 kB ( 0%)
initialize rtl : 0.01 ( 0%) 0.00 ( 0%)
0.00 ( 0%) 12 kB ( 0%)
address lowering : 0.03 ( 0%) 0.00 ( 0%)
0.01 ( 0%) 0 kB ( 0%)
rest of compilation : 20.06 ( 1%) 0.14 ( 0%)
19.97 ( 1%) 334344 kB ( 1%)
remove unused locals : 3.01 ( 0%) 0.01 ( 0%)
2.53 ( 0%) 0 kB ( 0%)
address taken : 1.20 ( 0%) 0.00 ( 0%)
1.26 ( 0%) 0 kB ( 0%)
repair loop structures : 0.18 ( 0%) 0.02 ( 0%)
0.15 ( 0%) 27 kB ( 0%)
TOTAL :1922.57 129.36
2051.17 38131807 kB
Command being timed: "gcc a.c -c -o delete.o -ftime-report -O3 -march=native"
User time (seconds): 1993.44
System time (seconds): 131.59
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 35:26.44
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 14667556
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 4316307
Voluntary context switches: 11113
Involuntary context switches: 215465
Swaps: 0
File system inputs: 65536
File system outputs: 1238168
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
