About JIT compiling

I think JITing has a ton of potential, there's alot of extra optimizations that can be done once you know the exact type of machine ur compiling for.

I don't think the lack of JIT speed is really a theoretical problem, but rather a practical problem. Compiler writing is hard, and not alot of ppl doing it. So I'd imagine I'd take a while for changes to 'come down the pipe' so to speak. And its more than just copying and pasting old static compiler code I'd imagine.

An fast JITter has little in common with a modern optimizing compiler. I suspect that Microsoft have decided to focus on developing efficient "JIT" algorithms and heuristic, and thus squeeze better performance out of the bulk of the code, rather than to write what would amount to an entirely different back-end for heavy-duty compilation. As long as you've got native code or even assembly code to fall back on for the most time-critical innerloops I honestly don't think this is a big deal. Still, I see no reason why they couldn't at least crank up the inlining depth in ngen.

Besides, if there's one thing Microsoft have proven unable to do well in past it is running unintrusive "background" tasks.

What you're describing is called ahead-of-time compiling, and it does happen. However, some languages make this harder because of reflection, dynamic module loading, etc and large parts of the program may not be visible until runtime. Many optimizations are also done interprocedurally so this makes optimizing the code you do have ahead of time less optimal.

The issues with regular JIT are that extensive optimizations are expensive; we're talking megabytes/gigabytes of data and minutes/hours of compilation time. It's just not feasable to do that in a JIT context if you're going to starve your actual application, nor is it worth it for an end-user application if you have to run it for an hour before you see any gains, and by then the application might not even be executing anywhere near the code you just finished optimizing.

Thats why most JITs will recompile methods and call chains multiple times over the course of a program. If a function is hot, especially if it's a leaf, it will get compiled at low opt. If it's still hot it will get recompiled at higher opt. If a chain of calls is scorching the compiler will inline where possible and apply the interprocedural heavy hitters to the remainder.

Having said that, JIT compilation has been blown way, way out of proportion. Optimization is an expensive game, and there are so many opts and passes that having to do it at runtime presents a lot of difficulties. I don't think JITs will catch up to static compilation in this regard any time soon, especially since profile-guided optimizations give you a lot of the same benefits as JIT.

The process of dynamic compilation in languages designed for it is trivial. Unlike C or C++, which need to do pre-processor and linking passes over large code-base, generation of code from C#, Java, &#106avascript, Lua, ... source is cheap.<br/><br/>Many safe optimization methods can be applied directly within function or class context. Multiplication by constant or 1, operations on constant, branches with constant conditions, and many more.<br/><br/><a href="http://jakarta.apache.org/bcel/">BCEL</a> is a project aimed at this and similar optimizations. Peephole optimization looks at code outside of context, and tries to discern invariants.<br/><br/>But no matter how tempting the though of data-driven optimization is, few safe optimizations can be made. For example - read an int between 0 and 250, display element at that index. Bounds checks may be removed, but what happens if entered value (declared as 8-bit) is in 251-255 range? One optimization is to avoid bounds check by making array 256 elements long, yet this alters the logic.<br/><br/>The theoretical optimization of removing bounds checks is no more accessible to JIT than it is to static compilation. Template meta-programming allows for truly efficient approach to these problems, but unfortunately, it has its limits, and is cumbersome at best.<br/><br/>But at the end of the day, JIT itself doesn't matter much. Bottle-necks are always few and determined through profiling. Greatest gains are made by algorithms. Conservative use of memory helps keep GC times down.<br/><br/>In reality, CLR or JVM are surprisingly fast. An equivalent application coded in C++ (GC or smart pointers, interfaces/pure virtual classes) will have same performance characteristics, since static compilation offers no benefits. If anything, managed languages do those things much better, since that's their primary function.<br/><br/>Perhaps the most interesting venue for optimization are functional languages. Those are typically much clearer on what the code is doing. Unlike C-family, where an iteration is using arithmetic on some variable and executing some more code, that may or may not be iteration (how many ways can you write a for loop). Declaring concept of iteration at language level however enforces design of algorithm, and compiler gains more knowledge about it. Current compilers approach it by trying to guess what the for loop really does.<br/><br/>But as said, right now, for the purposes VMs are used, these type of improvements do not matter enough, or the gains made are not noteworthy.<br/><br/>We are however moving into era of virtual machines precisely for that reason. The absolute time a single opcode takes to execute is quickly being dwarfed by current advantages of abstract machines.<br/><br/>When it comes to performance of VMs, two interesting projects to watch are Lua and currently very popular WebKit, both of which lead to many articles on various topics and issues involved.<br/><br/>PS: The above does not necessarily apply to JIT in particular, but at VMs in general