Time for another update. We finally started working on shading but before we get into that I want to talk about some of the other (more extensive) work we've done.

The Heap:

Let's start with the heap. This heap in this code is not simply “the heap”. It implements smart pointers with reference counting, bucketed objects, small 32 bit relative pointers and even some containers. This is where most of the last few months work went. Up to now the heap has been largely non thread-safe, although there were a few thread safe constructs. This was because it wasn't necessary. Our data searching structure (ghost tree) was used in a thread pool, however each thread in the pool had it's own section of the heap. This worked great since it made the threads completely independent. Since our data searching was the slowest part of the code the result was to speed everything up by a factor 6 or so.

However ….. once the slowest phase of the code was sped up to acceptable levels, we found that the second slowest part, started to stick out. This was the actual voxel generation. In some cases this was now approaching the time it took for data searching . The problem is that this was not so easy to multi-thread, since voxels have shared walls edges and corners, and if this code was threaded, not only the heap would have to be thread safe, but the objects themselves would need some sort of protection.

We decided to bite the bullet and add full thread safe capability to the heap, in addition to options for protecting individual objects. Voxel walls and edges needed to have mutexes since these could be subdivided, when any voxel that uses them is subdivided, and voxels would now be subdivided in parallel. In addition their reference count needed to be atomic. Shared object protection was achieved by adding optional mutexes to objects. These can be set to automatically be used by special kinds of pointers that reference child objects of a sub-dividable objects, such as a voxel faces or edges. All these features are optional so the ghost tree code still uses non-thread safe objects for speed.

Finally we finished converting all relevant objects over to our client / proxy object layout since it facilitates the use thread safety features.

The Mech Loop:

Since this will be a space game I want planets to orbit suns and moons to orbit planets and so forth. I decided to break this functionally into a separate thread (astral mechanics thread) we call the mech loop. For each iteration of this loop we go down the object tree and do time based movement to objects referenced by mech loop active references. This processes is now separate from the regular render thread and we have implemented the appropriate thread synchronization.

Chunk LOD transitions:

We have now implemented a MUCH simplified marching prisms (or cubes) LOD transition routine. We no longer tessellate voxels at all. Instead we simply tessellate the voxel faces in 2D where needed. We can then generate mesh edges along our faces. From there we use something akin to the naive surface nets algorithm and average all edge crossings to get a point in the middle to generate our triangles. Finally we go through a step to break up any non-manifold geometry we may have created. This is rare so we don't have to do it too often.

Shading:

So far we have only done some nominal shading. This is procedural in nature using 4D noise. The 4th dimension is used for altitude and we compress it to get a kind of layered effect. We will later add more detailed shading. We are also using a gaussian function to smooth out our normals for lighting calculations. We took all projection calculations out of the matrixes and now do them separately. We found the best solution was to divide the Z values by some large power of two in the vertex shader, to put them into the range expected by directX. This eliminated a lot of Z fighting and other numerical instability we were getting at long range.

Lights are objects and can be attached to other objects through references just like any other object in our system. They can therefor move freely. We do need to handle them a bit differently though, since the rest of the rendering is based on their position. We therefor do not render in our pass though the object tree, but build a command list and record all the light positions. Commands go to a linear chunk of memory so they can be executed in a cache friendly manner.

Here's the latest update video.