Distcc

[strook]

Ever tried distcc?
Great tool!
it improves the compile speed by a factor of 2 or more!

[klauss]

Wow... cool...

[breese]

it improves the compile speed by a factor of 2 or more!

distcc is mainly useful if you have more machines to compile on.

If you simply want to use all the cores on a single machine, then use the "make -j2" command (replace 2 with the number of cores your machine has.)

[strook]

maybe (i haven't tried out it yet) you would maximize the speedup on for example a machine with 2 cores with -j3, cause the processor makes often noops cause the pipeline is not full, cause of data dependencies (code that must be executed in serial order).
but on modern cpus there are several workarounds.
but maybe you get a speedup if you have a very long pipeline and many execution units on the cpu.
today the cpus are optimized in trying more and more parallel.

[klauss]

it improves the compile speed by a factor of 2 or more!

distcc is mainly useful if you have more machines to compile on.

I know... hence

Wow... cool!!



But ya, I don't have several computers at my disposal atm

But at home, ya, I do have about 3. An old one, an even older one, and a "mature" one. Perhaps together they can make one "not so old" one

I bet my energy company will disagree

[klauss]

maybe (i haven't tried out it yet) you would maximize the speedup on for example a machine with 2 cores with -j3, cause the processor makes often noops cause the pipeline is not full, cause of data dependencies (code that must be executed in serial order).

Well, not true.

In order to get advantage of that (you're talking of hyperthreading), your processor actually has to support hyperthreading (core 2's don't for instance), and furthermore, your tasks have to be taxing different execution units in the CPU. Which is rare in compile workloads.

Compilation is usually memory-bound, and even with 4 cores, a memory-bound process with -j4 won't perform at 4x. AMD processors cannot use the entire bandwidth of the memory bus on one core, so yes, -j4 may work nicely on a 4-core CPU, but -j5 will probably just increase cache pressure and be counterproductive. On intel processors, however, single-core memory bandwidth is much better, and 2 cores can easily saturate a 4-core's memory bus. So -j4 would be hard-pressed to outperform -j3, say.

The thing is rather complex, so a good rule of thumb is -jn with n=number of processors. Going over it is dubious, and a smaller number is probably worth a try.

So... with complex things as these, better not speculate and benchmark instead. I know for instance HT alone does shave off about 5-10% of build time on a P4 I have with -j2. So it's kind of worth it. And -j4 on i5 works wonders. -i8 on i7 is the same, HT in i7 is a whopping lot better than on the P4, it really performs a lot better, but still, the i7 is suffering from cache pressure already with -i8, so I usually use -i6 there.

[strook]

compiling a linux app on macosx didn't work,
there were too much linker errors.
also xcode does multithreading in compiling already.
speedup in compiling would be also only if i buy a second mac.

[strook]

I noticed, in eclipse you can too enable parallel build when Using a makefile imported project.

[pheonixstorm]

While i'm thinking about it.. There is (or was) a linux livecd that had this feature. I used it extensively when building my gentoo boxes since I liked doing the stage 1 build from scratch