Ever tried distcc?
Great tool!
it improves the compile speed by a factor of 2 or more!
Distcc
-
- ISO Party Member
- Posts: 461
- Joined: Fri Sep 03, 2010 12:10 pm
Distcc
plz visit my vegastrike project branch here
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
-
- Elite
- Posts: 7243
- Joined: Mon Apr 18, 2005 2:40 pm
- Location: LS87, Buenos Aires, República Argentina
Re: Distcc
Wow... cool...
-
- Bounty Hunter
- Posts: 152
- Joined: Thu Sep 02, 2010 8:00 pm
Re: Distcc
distcc is mainly useful if you have more machines to compile on.strook wrote:it improves the compile speed by a factor of 2 or more!
If you simply want to use all the cores on a single machine, then use the "make -j2" command (replace 2 with the number of cores your machine has.)
-
- ISO Party Member
- Posts: 461
- Joined: Fri Sep 03, 2010 12:10 pm
Re: Distcc
maybe (i haven't tried out it yet) you would maximize the speedup on for example a machine with 2 cores with -j3, cause the processor makes often noops cause the pipeline is not full, cause of data dependencies (code that must be executed in serial order).
but on modern cpus there are several workarounds.
but maybe you get a speedup if you have a very long pipeline and many execution units on the cpu.
today the cpus are optimized in trying more and more parallel.
but on modern cpus there are several workarounds.
but maybe you get a speedup if you have a very long pipeline and many execution units on the cpu.
today the cpus are optimized in trying more and more parallel.
plz visit my vegastrike project branch here
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
-
- Elite
- Posts: 7243
- Joined: Mon Apr 18, 2005 2:40 pm
- Location: LS87, Buenos Aires, República Argentina
Re: Distcc
I know... hencebreese wrote:distcc is mainly useful if you have more machines to compile on.strook wrote:it improves the compile speed by a factor of 2 or more!
Wow... cool!!
But ya, I don't have several computers at my disposal atm
But at home, ya, I do have about 3. An old one, an even older one, and a "mature" one. Perhaps together they can make one "not so old" one
I bet my energy company will disagree
-
- Elite
- Posts: 7243
- Joined: Mon Apr 18, 2005 2:40 pm
- Location: LS87, Buenos Aires, República Argentina
Re: Distcc
Well, not true.strook wrote:maybe (i haven't tried out it yet) you would maximize the speedup on for example a machine with 2 cores with -j3, cause the processor makes often noops cause the pipeline is not full, cause of data dependencies (code that must be executed in serial order).
In order to get advantage of that (you're talking of hyperthreading), your processor actually has to support hyperthreading (core 2's don't for instance), and furthermore, your tasks have to be taxing different execution units in the CPU. Which is rare in compile workloads.
Compilation is usually memory-bound, and even with 4 cores, a memory-bound process with -j4 won't perform at 4x. AMD processors cannot use the entire bandwidth of the memory bus on one core, so yes, -j4 may work nicely on a 4-core CPU, but -j5 will probably just increase cache pressure and be counterproductive. On intel processors, however, single-core memory bandwidth is much better, and 2 cores can easily saturate a 4-core's memory bus. So -j4 would be hard-pressed to outperform -j3, say.
The thing is rather complex, so a good rule of thumb is -jn with n=number of processors. Going over it is dubious, and a smaller number is probably worth a try.
So... with complex things as these, better not speculate and benchmark instead. I know for instance HT alone does shave off about 5-10% of build time on a P4 I have with -j2. So it's kind of worth it. And -j4 on i5 works wonders. -i8 on i7 is the same, HT in i7 is a whopping lot better than on the P4, it really performs a lot better, but still, the i7 is suffering from cache pressure already with -i8, so I usually use -i6 there.
-
- ISO Party Member
- Posts: 461
- Joined: Fri Sep 03, 2010 12:10 pm
Re: Distcc
compiling a linux app on macosx didn't work,
there were too much linker errors.
also xcode does multithreading in compiling already.
speedup in compiling would be also only if i buy a second mac.
there were too much linker errors.
also xcode does multithreading in compiling already.
speedup in compiling would be also only if i buy a second mac.
plz visit my vegastrike project branch here
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
-
- ISO Party Member
- Posts: 461
- Joined: Fri Sep 03, 2010 12:10 pm
Re: Distcc
I noticed, in eclipse you can too enable parallel build when Using a makefile imported project.
plz visit my vegastrike project branch here
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
plz support VegaOgre by donating to it!
My systems: Mac mini 1, 4gig RAM;
i5 Quad Core 2400, 300mbit WLAN, 1,3Tbyte HD, 60 GB SSD,
nvidia geforce 8400gs 512MB, 6gig RAM with Ubuntu 11.4,
win7 and hackintosh installed
-
- Elite
- Posts: 1567
- Joined: Tue Jan 26, 2010 2:03 am
Re: Distcc
While i'm thinking about it.. There is (or was) a linux livecd that had this feature. I used it extensively when building my gentoo boxes since I liked doing the stage 1 build from scratch
Because of YOU Arbiter, MY kids? can't get enough gas. OR NIPPLE! How does that mkae you feeeel? ~ Halo