On 2007/11/27, Vladimir Vassilev <vladimir@xxxxxxxxxxxxxx> wrote:
> J.C. Pizarro wrote:
> > It's the same problem of "Nine Woman Can't Have a Baby in One Month"
> >
> > "Nine Cores Can't Reduce a Compilation of 9 Seconds to One Second"
> >
> In case the project is composed of reasonably sized files and the number
> of those files is sufficient the parallel build gives good scalability.
> The "make -j" option is a very powerful tool not only in terms of
> compilation. We have been using this also for rendering and other time
> consuming jobs. It basically allows you to build in parallel any target
> described in a Makefile. Implementation of MPI based plugin module for
> GNU Make (the -j option comes with a well defined interface to spawn the
> "threads" and it is easy to implement that for any MPI or even socket
> interconnected number of hosts) in combination with shared file system
> has allowed us to build various time consuming projects on computer
> clusters gaining almost linear scaling.
>
> Vladimir
About scaling, we can use LiveCDs to test the measures of compile-time
in clusters or virtual clusters of machines with not much effort.
These LiveCDs can be ParallelKnoppix, Knoppix OpenSSI, Clusterix,
ClusterKnoppix, Scientific, Quantian, Rocks Cluster, VMKnoppix, Xenoppix, ..
On 2007/11/27, Sven Eschenberg <eschenb@xxxxxxxxxxxxxxxxxxx> wrote:
> I am not sure about ccache, but I thought it does some file and
> preprocessing caching (not exactly sure, how it works, I thought, it kinda
> gets called instead of the preprocessor or at least before the PP).
Combining distcc & ccache is little complicated because of the square vision
with two triangles in the fourth vertex:
no distcc ---------------------------- ccache-only
no ccache |
| |
| |
| \ 1st ccache of the local machi$
distcc-only ---------------- 1st distcc \ 2nd distcc (otherwise)
2nd ccache of \
the remote machine \
How to redistribute to the machines the ccache's data?
The quick and dirty strategy is to use "rsync" between every dirs
/ramdisk/tmps/ccache_data/ of the participing remote machines.
It's replicating and merging the local data of ccache of each local
machine to every machines of the cluster.
I suppose that doesn't exist yet a "distributed distccache for GCC".
J.C.Pizarro
