Why is compiling so slow?

Look at pre-compiled headers...

Also,

- Make sure you haven't got any headers included in header files that you dont need.

- Make sure you are not including headers when a forward declaration of a type will do.

Uhh, to be honest I include EVERY Header in EVERY source file! It makes programming so much easier ^^


Thats how I do it:
I have one Header called AllHeaders.h. In this header there are zillions of includes (one for every header in my project). Now in every cpp file I only have to include AllHeaders.h and get everything I need.

May this be the problem?

But wouldn't it be waaaay to much work, to individualize the includes for every cpp file?

Quote:

Original post by VanKurt Hi guys! I'm working with VisualStudio 9 (Orcas) on a medium-fast PC (only singlecore). Compiling my game (which isn't too big) takes more than one minute. And this can be quite annoying!! So I asked myself why compiling takes so long. Is my PC not fast enough? Would a Quadcore speed it up? Is there something wrong in my software setup? Or is EVERYONE suffering from the same problem?

I'm assuming your game is written in C++. Orcas provides some multi-processor build enhancements. Also make sure you have enough memory so that your build doesn't cause excessive swapping.

However, in the end it really depends on your project size. I'd love to be able to build in only a few minutes; I've experienced builds lasting anywhere from 15 minutes to several hours at work.

Edit: The posters before my provide valuable advice. Reorganizing your #includes should definitely help.

Is there a way to display build times in VS9?
That would make optimizing easier...

Quote:

Original post by VanKurt May this be the problem?

Yes. Pretty much guaranteed.

Quote:

But wouldn't it be waaaay to much work, to individualize the includes for every cpp file?

No. Each CPP file should only be dealing with code from one related set of functions anyway; one related set of functions will generally all need the same headers.

You're right that the whole include mechanism sucks, though. Other languages, like C#, don't have it.

Hey,

I experienced the same problems on the last project that i did, and it got to the stage where compile times were very long. Since then i have read about what a few posters above have mentioned regarding where to #include and it does make a huge difference. Does anyone know of an article on how to organise multiple c++ header and source files? Hopefully it maybe also helpful to the origional poster.

Thanks

Does something like ccache exist for VS?

Organizing Code Files in C and C++.

*sigh*

I really need to finish that article I've been working on that illustrates just how this practice of "master include" files can cause seemingly illogical compiler errors, unintended compile-time bloat, and other nastiness.

Perhaps this weekend.

Is there a way to find out which headers a cpp file needs?
In my project there about 30 Headers and most cpps will need about half of them.

So testing around to find out which headers must not be included is a lot of work...
But I'm sure this could be automatized ;-) (Is there a build in function for that?)

Quote:

Original post by VanKurt Is there a way to find out which headers a cpp file needs? In my project there about 30 Headers and most cpps will need about half of them. So testing around to find out which headers must not be included is a lot of work... But I'm sure this could be automatized ;-) (Is there a build in function for that?)

Firstly, be aware that much of the time you don't need a full header. If you're just referring to a class through a pointer or reference, you don't need its full definition; a forward declaration will do.

Also, you should aim to satisfy any dependencies your headers have inside the headers themselves; if <a.h> uses stuff from <b.h>, then it should include it - it should not rely on you putting <b.h> before <a.h> in any files where you actually want to use it. It means you can then just include <a.h> without having to know anything about other headers that it depends on.

Anyway, if it really is a big thing, then you probably want Lint.

Quote:

Original post by VanKurt Uhh, to be honest I include EVERY Header in EVERY source file! It makes programming so much easier ^^ Thats how I do it: I have one Header called AllHeaders.h. In this header there are zillions of includes (one for every header in my project). Now in every cpp file I only have to include AllHeaders.h and get everything I need. May this be the problem? But wouldn't it be waaaay to much work, to individualize the includes for every cpp file?

Eeeeew, seriously, don't do that.

We had similar problems on my current project. You'd change a header file for "Enemy A" for example and because that was included somewhere that was included somewhere else and so on it'd end up compiling 250+ files. I spent a few hours reorganising everything, only including when needed including and got that down by loads. In many cases only single figure files needed to be recompiled after changing headers.

If we may veer into the slightly more theoretical, ignoring possible compiler optimizations, the fact that each C++ translation unit (and this goes for C as well) is fed to the compiler independent of all others makes C++ compilation slower than it needs to be. Basically, the C++ compilation process looks like this:

1. Take an implementation (.cpp) file. Collate all the headers it includes, and the headers they include, and so on. Concatenate each header in place, creating one large .cpp file. Headers will be repeated within this large file unless "inclusion guards" are used.


2. Compile this one large .cpp file to produce an object (.obj, .o) file.


3. Move on to the next implementation file, possibly performing header preprocessing, symbol creation and so forth for the same header files again if included in multiple implementation files.

Why does C++ do this? Why doesn't it cache symbols and declarations/definitions it gleans from processing header files? Well, the reason is something called the forward declaration requirement, which states that an entity must be declared before it is referenced. The forward declaration requirement itself is a side-effect of a single-pass compilation model - C was designed such that a source file could be preprocessed and compiled in basically a single pass, with header files being evaluated and concatenated in the order of their corresponding #include statements, and C++ inherited this behavior for legacy reasons.

(Modern C++ compilers do a lot of things to mitigate the impact of this model, but they are still slaves to it to a large extent, mandated by the billions of lines of existing source code and thousands of libraries already in production, as well as by certain provisions of the language standard. A legacy-free C++ compiler could dispose of header files entirely, though I haven't applied much thought to what impact that would have on templates.)


Anyway, the point is that C++ compilation is slow pretty much by definition. You can use clever hacks like pre-compiled headers (which are a compiler extension to create an object file that is a concatenation of all the object files containing definitions for entities declared in the headers specified, eliminating the need to recompile those translation units and simplifying the link step by requiring fewer relocations), but it'll pretty much always be slower than a language engineered with compiler-friendly characteristics (like, allegedly, Haskell, with which I have no personal experience).

One minute build time is not too bad. Wait til you get on a big project where you're dealing with build times of 10 minutes or more for a complete rebuild.

Ive known a rebuild time of over two hours in a previous job. That was not fun.

One thing that takes a *lot* of time with big solutions is calculating project dependenties. I think thats been improved, but its still anoying. Another thing is that there is usually a big overhead, in addition to processing headers, for each source file you send to the compiler. This means for the same number of lines of code, the less .cpp files you have, the faster you will compile. There's an obvious way of speeding this...

Templates also take their share, but I love templates, they are 100% worth it.

The first thing of course is to learn to use forward declarations. Never include an head in an header unless it is absolutely needed.

edit: I dont think anyone mentionned it yet, but if youre on a network, Incredibuild is a lifesaver.

At my work we mitigate this problem by creating what we call "unity builds". Essentially, you generate your .cpp files and .h files normally, but you only compile one (or a few more depending on the size of the project) master .cpp file that includes all of your normal .cpp files.

This gives a huge boost to compilation speed; generally better than any other method.

There are problems, of course.
* Because all of your code is in one translation unit, anonymous namespaces/static functions+variables don't work. That is, they don't function as you'd expect.
* It's difficult to track down an unresolved external.
* It's difficult to tell from where an identifier is being included.
* #defines will leak through your files. You have to #undef at the end of files if you use local #defines.
* Expect order-of-inclusion problems.
* If you ever use the code you write in a project without unity builds, expect to have to add many missing includes.
* You have to remember to add new .cpp files to the unity .cpp (or autogenerate it).
* Compiling a single file often won't work.
* Building uses a lot more memory.

Some of these things are true without unity builds, but are much worse with them.

You might be able to break your code up into multiple libraries. That's another way to organise things.

Quote:

Original post by ennuija At my work we mitigate this problem by creating what we call "unity builds". Essentially, you generate your .cpp files and .h files normally, but you only compile one (or a few more depending on the size of the project) master .cpp file that includes all of your normal .cpp files. This gives a huge boost to compilation speed; generally better than any other method.

Actually, it increases compile times but the linking times drop hugely because there are a smaller number of object files therefore the overall build time goes down.