Compile Time. How to reduce it?

I still have many thins to implement and learn but the compile time of over 3 minutes is killing me.

Lucky you. Our compile times are over an hour.

So my question is: How to reduce it?[/quote]
If you need fast compilation times, C++ is the wrong language. That said, I assume that you are too far into the project to switch languages, so:

Avoiding dependacies and reducing #include/s? Yeah but how? I tried to do it however the time still doesn't change. Even if I redesign some of the parts. [/quote]
Only include the absolute minimum necessary files in each header. Eliminate all circular dependencies between your own header files (forward declare everything possible instead).

Any tips of splitting files in multiple projects so I can enable multicore compiling? How shoud I spread them. Where is best to keep the file with interfaces. And finally does it worth the pain of doing it?[/quote]
You can split your game up into multiple projects, by layer. I like one project for external dependencies, one for the low-level support code, one for the 'engine' code, and one for the game itself. This way each time you make a change, you only have to recompile the changed project, and the projects 'above' it in the hierarchy.

Any other suggestions?[/quote]
Include external library headers into a single pre-compiled header file.

Understanding how compiler works if the best way to go.

most common method;
- Define your translation units, make a list of what each of them includes.
- Find headers that are included more than once.
- If these headers rarely change put them in your precompiled header. ( if you dont know what this is, better google it)

use #pragma message("including XXX.h") to find a file is include more than once.
remember header guards and '#pragma once' does not help if same file is included in more than one translation unit.
ex. if you see an output something like this, then header.h is a good candidate for precompiled header
compiling a.cpp
including header.h
compiling b.cpp
including header.h

other tips include;
- try your best to not include a header file in an other header file. use forward declarations if possible.
- avoid big header files.
- header files should change very very rarely. a change in an header file will cause all translation units including it to be recompiled. designing you classes before writing them is a good idea.
- if your project is modular and a module is stable, make that module to its own project.

Some quick fixes: Put your code directory on an SSD. Add exclusions for that directory to your virus scanner. Don't make millions of small CPP files, bigger ones can be faster.

Enable incremental-everything. This can be tricky to get working in MSVC... which compiler are you using?

Run header hero over your code to find problem headers.

If you're desperate: Try unity builds.

Advanced: Use procmon to see what the bottlenecks are.

Our compile times are over an hour.

You're not serious...!? I'm assuming that's for a full rebuild of multiple platforms/configs, as opposed to an incremental build of a single target (and also not on a distributed build server)?

Ohh sorry. Yeah I'm using VC++ 2010.
Thanks for the advices. Going to read the links now...

Hmm,
I have seen some larger Projects where a lot of developers was involved. A smart but effective way is splitting the whole Great Project in smaller Statically Lib Projects. (*.lib) wich can single compiled. Then you only have generic Base EXE Stub where you should link against with and wich is the Host of your Lib Project.For example: If you compile
tthe Subproject (for exampe physics.lib or render.lib) you can simply build run/debug it within seconds. This is why you only have to build you Lib Project.
Of cause. If you compile all you Projects, this will takes a long time if your Project is relly hugh, but working with small subprojects dramaticall brings
down you effective working cycle recompiletime. It is also recommended in large projects to have an Buildserver wich periodically builds the whole project drivven
by a Taskmanager or Cronjob on an diffrent machine.At the End you have a Single Exe build from various subprojects. This works like a charm for Exe related
things. For other, more generic Suprojects it might be better to build some DLL (or on mac Linux SO - Shared Objects) to avaoid completly monolitic structures
and be more modular.

If you use VC++ make sure you use WIN32_LEAN_AND_MEAN as preprocessor makro and if possible Precompiled Headers, this saves also much compiletime
and make realy sure to use Include Guards to avaoid multiple inclusion of the same Headers. This can also cause strange Linker Errors wich can you cause extreme headache to solve.If you use an GCC Compiler try the compiler cache. At debugging you can use the reply feature for modify values, reply and got ahead, so you are not forced to recompile every 30 sec. for debugging purposes only.

Peter

There is another obvious solution: get a faster PC. Something with many cores will speed up compilation a lot.

How many lines is it? My project is 34000 c++ lines and takes less than 11s to recompile from scratch on a 2 year old PC (no pre compiled headers).

The common tricks are:

1. Reduce dependencies, both in terms of headers and libraries
2. Reduce complex template usage (this depends a lot on the compiler)
3. Turn off link-time code generation/whole program optimizations for non-final builds
4. Use pre-compiled headers
5. Use a distributed build system (only applicable if you're on a network with lots of PC's that you can install the client on)

For MSVC 2010 another option you can try is to use the /MP switch to enable builds on multiple cores if they are available on your machine. Obviously how much benefit you get is hardware dependent.

SSD's and more CPU cores have already been mentioned. Another, cheaper solution would be to create a virtual file system in RAM (easy and quick to mount a tmpfs in Linux, 3rd party needed for Windows). I used to mount a tmpfs when compiling larger code bases (the dreaded boost source, for instance) when using traditional hard drives. Today, I have installed SSD's in all my machines, and even my i7 @ 5 GHz cannot saturate this bandwidth enough for a tmpfs to reduce compile times much further.
TL;DR: Using a FS in RAM (tmpfs) can also be a solution if you are I/O bound.

There is another obvious solution: get a faster PC. Something with many cores will speed up compilation a lot.

How many lines is it? My project is 34000 c++ lines and takes less than 11s to recompile from scratch on a 2 year old PC (no pre compiled headers).

WTF?!?!?! 34000 lines? I've got somwhere around 10000-15000 and my compile time is 3 min on a laptop with i7 4x 2.20Ghz. I guess I'm writing really inefficient code ;/
Warning 1 warning D9030: '/Gm' is incompatible with multiprocessing; ignoring /MP switch. Ok, I'll remove /Gm(I don
t even know what is it about , need to google it first)

Edit: I'm feeling like the flash with /MT . Compile time got reduced 5-6 times. Also I have already found some headers I rarely edit however they are included from many places. I think they are candidates for precompiled headers.

I've got somwhere around 10000-15000 and my compile time is 3 min on a laptop with i7 4x 2.20Ghz.

I am using Linux, and that may be different. But I think the Windows environment should be possible to get to the same speed. Anyway, I am happy it seems the /MP switch solved it for you!

I read somewhere Linus Torvalds claimed recompiling the Android kernel wasn't a problem at all. He used 32 threads...

Actually I love linux, too(I've got ubuntu installed here too). However I wanted to learn the DX API..
It seems that for now /MT will do the job . However I'll try the other approches because soon I may get long compile time again..

C++ compile times can also massively depend on libraries used. You can easily get minute long compile times on low end hardware from a single relatively short .cpp by using something like Boost.Spirit. But then again if you were to write all that code yourself it wouldn't compile for free either ;).

[quote name='swiftcoder' timestamp='1347120461' post='4978009']Our compile times are over an hour.

You're not serious...!? I'm assuming that's for a full rebuild of multiple platforms/configs, as opposed to an incremental build of a single target (and also not on a distributed build server)?[/quote]
It's a full rebuild, yes, but only for a single platform/config. Incremental builds are on the order of several minutes, but due to various inadequacies in the build system, one ends up doing a full rebuild several times a day...

What about just not recompiling object files that are already compiled and still in date? Wouldn't that save some time?

Gonna try the RAM disk suggestion - I should put both the code AND the compiler on the RAM right? (But the IDE can stay on disk?)

I am using Linux, and that may be different. But I think the Windows environment should be possible to get to the same speed. Anyway, I am happy it seems the /MP switch solved it for you!

I read somewhere Linus Torvalds claimed recompiling the Android kernel wasn't a problem at all. He used 32 threads...

How do you go about setting up GCC to take advantage of multithreading? My project is much smaller than yours, but takes about 15-45 seconds (3-5 minute full builds) to do an incremental build. (I use MinGW). I've passed in what I thought was the correct flag, but I can't tell much of a difference.

I use alot of heavy DLLs though (Qt), but that shouldn't affect compile time (being dynamically linked) should it? Qt isn't very template heavy and uses pre-declarations for almost everything.

How do you go about setting up GCC to take advantage of multithreading?

GCC doesn't really support multi-threading (in the sense you mean). But 'make' does, so you can feed it the -j N option to spawn N different compile tasks at once.

If you have the $$$ you should give IncrediBuild ( http://www.xoreax.com/ ) a try, I've used it for years in my professional jobs. It basically saved a few projects, like swiftcoder we also had 1+ hour compile times until we switched over to using it, then it reduced down to 5 minutes full recompile and <1 minute normal case.

I didn't read trough the whole thread, sorry if someone already answered this. You obviously use header protection #ifndef or #pragma so the contents of the header is only included once. Lets say you have the following two lines in the same file (unlikely, but entertain me)
[source lang="cpp"]#include "header.h"
#include "header.h[/source]
The contents of header.h was included once, but the header.h file was opened twice (expensive). the easy (and ugly to read) workaround for this is:
[source lang="cpp"]#ifndef _H_HEADERH_
#include "header.h"
#endif
#ifndef _H_HEADERH_
#include "header.h"
#endif[/source]
Assuming _H_HEADERH_ is the #ifdef guard defined in header.h. This is an oversimplified example, but you can see how it helps. Back in my college days adding this to my final project reduced our compile time from 16 minutes down to about 10.

Disclaimer, i now have an SSD macbook, and this trick does very little to help compile times.
Lesson: hard-disk reads are expensive when there is an actual disk involved.