Any recent improvements in LOD generation technology?

It's possible to do pretty well with just QEM edge collapse, with the right choice of cost function and a good implementation. The core algorithm of a dynamically-prioritized sequence of edge collapses is quite good. All implementations I've seen (aside from mine) suffer from a major deficiency: they enforce hard constraints on when edges can be collapsed, for example preventing flipping normals of triangles if the angle changes by more than 90 degrees. Hard constraints somewhat non-intuitively actually harm both quality and the ability to produce a well-simplified mesh.

Imagine you have a very detailed mesh with a noisey rough surface (typical output of 3D reconstruction). It can sometimes be necessary to flip a triangle more than 90 degrees temporarily from its original orientation. If you don't allow this to sometimes occur, and instead enforce normal orientation using a hard constraint, the algorithm tends to paint itself into corners which prevent further simplification.

It's much better to enforce constraints using a soft cost function which is updated every time a collapse occurs in a local neighborhood of the mesh. For example, rather than a hard normal-flipping constraint, it's better to add a term to the cost function which penalizes changes in the normal direction of adjacent triangles. Soft constraints allow you to exactly control the exact number of triangles in the output mesh, since you can just do collapses in increasing-cost-order until that number is reached. This is useful for LOD generation, where you might want to exactly control the number of triangles at each LOD level, getting the best possible quality/efficiency at each level.

Another common problem which seems to happen in your example is that the standard QEM formulation does not penalize edge collapses within the plane of planar surfaces. This makes it easy for flat geometry like your building to get “eaten away”. The QEM cost function calculates the cost of a target vertex as the squared distance to the planes of the adjacent triangles. If the target vertex lies in the planes of all of those triangles, then its cost will be very low, making it easy to produce a bad collapse. The solution to this is to detect sharp or disconnected edges, and to add extra “virtual” planes to the cost matrix which are perpendicular to the edge normal. This helps to prevent bad simplification in the edge normal direction (since that would now have a much higher cost).

However, there is still a limit to how much edge collapses can simplify while retaining good quality. The main limitation is the topology of the input mesh. If the input mesh is a simple closed manifold (e.g. like generated by isosurface extraction of an SDF), then it works well. However, if the mesh is built from many separate parts, this tends to not simplify as well because the mesh connectivity is disjointed. Edge collapses can't usually remove holes in a mesh (changing the topological genus).

The best and most common solution to this is to just get the artist to manually create simplified versions of all meshes where edge collapsing doesn't work well. The artist can then manually omit building details, replacing the entire mesh with a simple box. To do such a thing automatically and have it always look right for all kinds of inputs would be very difficult.

Re-meshing algorithms can also help. This could involve converting the input mesh to a SDF, then extracting a manifold isosurface at the desired resolution (using the “Manifold Dual Contouring” approach), possibly followed by edge collapses. This will make it possible to merge disjointed parts of the mesh and get a better quality simplification. However, keeping textures consistent after remeshing might be hard.

Here are some quick examples of my QEM edge collapse on Sponza, 262k triangles, 10k triangles, 1k triangles. 10k triangles still looks pretty good. At ~3k it starts to fail and mesh gets eaten away. Texture coordinates and normals are handled reasonably well. I could probably improve the handling of the cost function on sharp manifold edges to better preserve the mesh silhouettes with fewer triangles.

Original 262k triangles:

10,000 triangles:

1,000 triangles:

Triangle count is not what you should benchmark your LOD by. What you really want from a LOD is to ensure that triangles cover 2×2 pixels or more, and that you get maximum area to outline ratio for triangles. The cost of vertices themselves is almost nothing, it is the shading of small triangles that is expensive. And you want to stay as true as possible to the initial outline.

Nagle said:
What's going on with LOD technology? Any good systems where you put in a textured mesh, and you get out a simpler textured mesh with surface geometry reduced to normals in new textures? Anything where you can push this hard for very low LODs, < 100 triangles, and still have something that looks decent at distance?

Yes, but not like that. We're approaching the point where the reason for what used to be done has almost vanished. We're regularly in the 5-10 million triangles rendered these days and it isn't even a blip on the profiler, with far more triangles being dynamically processed and culled by the engine to the point we don't care about it.

Much of what you described are techniques from the 1970s and 1980s. Discrete LOD systems where a full mesh of one density were replaced with another mesh of a different density were all the systems could handle. Similarly with textures, discrete texture layers for mipmaps let it swap out.

Continuous, viewpoint-dependent LOD (CLOD) were big in the 1990s and early 2000's, and were what I researched a lot of in grad school. The systems would preprocess models and you'd use a mix of viewpoint-specific details to determine what density of mesh to use in that segment, with high priority to edges with strong tangents and low priority to surfaces seen straight on. Effects like quadtree morphing could smoothly transition patches, no popping or seams. Similarly with textures, back when I was in grad school it was more theoretic but we saw it more and more in AAA games there was a shift to layered composite images for textures, which I first used in games in 2007 though I had studied it back in 2002 from siggraph papers.

Next on meshes, hierarchical LOD (HLOD) systems were mostly theory in the 1990s, then transitioned to “unlimited detail” systems around 2000-2005. Shader-driven systems allowed this to shift into hardware but was mostly constrained by data transfer speeds. With fast SSD's becoming ubiquitous and NVMe commonplace, this really entered the consumer market in around 2020. On the last four titles I've worked on SSD was a requirement and NVMe drives recommended mostly for graphics purposes, the meshes are effectively streamed from disk into video memory for view-dependent high quality rendering. The real-world implementation, rather than theoretical, is with Unreal's Nanite system.

On the texturing scene, similarly shifting to shaders running dynamic ray tracing, dynamic global illumination, and dynamic reflection systems. There were some earlier experiments on it, but the shift from static to real-time raytracing (RTRT) papers blew up staring in 2005 and 2006 thanks to shaders. There were some experimental groups making real-time raytraced Quake. With RTRT lighting and shadows could also come RTRT-powered voxel shadows and lighting, which first started making the practical rounds in 2018 and 2019.

We're now at the point where mainstream games are using both HLOD for meshes and real-time raytracing for texturing and lighting. Unreal Engine 5 has made it as easy as a couple checkboxes. Throw in all the high detail meshes you want, and compose textures of high detail textures and parametric surfaces. During asset cooking meshes will have their hierarchical data generated, so it doesn't matter if the dragon is chomping down and you're looking at their bloodstained teeth and powerful jaws, or if you're looking across the valley at a tree-filled scene, you'll always have around 3-10 pixels per patch of mesh. There's no need to precompute the lighting and shadows, they will be computed live with adaptive shadow maps, flowing smoothly with animations. The shadow from a flag rippling in the distance, even though a portion may occupy less than a single pixel, still shifts the pale shadow's lighting with the wind animation of the distant flag.

The current industry practice for my last two AAA games and the current titles is that we generally don't need to worry about it. Main characters and bosses you'll get up close and personal with can easily cross 50K triangles but the art department is lax on it. As an experiment we had a 200K triangle dragon inside a 500K polygon cave filled with high resolution objects that wasn't even a blip on the performance metrics, even the older GT1060's and 1070's -- minimum for running VR headsets a decade ago -- handled it just fine. I tested it on an older machine (i7-4790K, 4GHz, GTX 970 video card, 16GB memory, 4GB VRAM) that had an old VIVE rig hooked up because I had it available and it ran in VR just fine, no stutters, which surprised me a bit. It's been a while since I looked at our charts but I seem to recall 1000 to 5000 tris was recommended for props and small objects but not enforced by our art teams, those just aren't the bottlenecks they were decades ago. I remember being surprised in Sims 3 where 512 tris was the recommended small size back in 2008 and 2009, the limits keep getting relaxed. As HDR displays are becoming more commonplace, more artists and art directors have studied how they work. (I first studied it for photography and bracketing multiple exposures then editing in lightroom, and its interesting for me to read up on the techniques and implementations.)

Textures are a somewhat different story, and we're sometimes pushing against texture limits on the games I'm on. But that's with most significant objects getting multiple 4K textures, typically 4-8 texture maps applied in their materials, with the art passes resampling to 1K textures or 512 where it the higher resolution doesn't really make sense for the screen space. Even so, most game's minspec systems these days have graphics cards with 3GB VRAM, the game engines can sort that out rather nicely.

In general it's more about shader complexity than triangles. For profiling passes, I see 5-10 million rendered triangles works well in a scene, and given what the HLOD system in Unreal provides, that's surprisingly difficult to hit because the system manages it so well. Unreal's Nanite system focuses on exactly the same critical points we were focused on during the CLOD terrain systems back in 1999-2002, keeping a sharp profile edge and approximately the same visual space with low visual error for each patch.

Way back when I started and first learned to program none of this would have been possible. When I was writing my first ‘big" graphics programs we absolutely needed to swap out geometries because of the hardware limitations. In 1997 I remember building for the high end 8MB VRAM computer systems didn't have room for fancy textures and many meshes. The Indy, Onyx ,and O2 systems in the graphics lab were pushed to their limits. All of those problems have been solved for over a decade, and nearly all game systems can handle them.

Nagle said:
Is anyone working in this area? New SIGGRAPH papers?

As for what's currently bleeding edge and not yet into mainstream, we've got multidimensional procedural wave textures, better adaptive fluid simulations, more photorealistic diffusion and translucency for skin and other translucent materials or emissive materials, and more cloth simulation details that can accurately render and animate different stitch types and material types across fabrics. All of them looked like siggraph 2025 papers that will soon be coming to games in the next few years. You can find links to a lot of the siggraph 2025 papers here.

Nagle said:
Here's the problem. This is content from the Second Life metaverse, simplified by a rather dumb LOD algorithm. The building has gone see-through.

The building seems a collection of instances of multiple modular models (windows, floors, ceilings, etc.).
Even if all those models are technically just one single triangle soup, LOD reduction algorithms usually have problems with reducing beyond a single model, because there is no adjacency to connect the parts.

Iirc, Simplygon can deal with such situations. It can also transform geometric detail to texture normals, merging multiple textures and materials into one, etc.
Reading the documentation it looked very impressive to me, even i could not find all those features in the free web version of it when i tried it out. Surely worth to check out.

Nagle said:
Is anyone working in this area? New SIGGRAPH papers?

How about Nanite? NV has a partial implementation on github for example.
It could render such scenes with ease. But notice that's not only due to LOD but also due to cluster based occlusion culling and software rasterization of small triangles. All this is a lot of work ofc.

Personally i've implemented the occlusion culling, which is little work and i'm very happy with it. It's worth it already for the Sponza scene.
I'm using the mesh optimizer library to make clusters, which can also do QEM reduction, and the dev also tries Nanite alike stuff iirc.
It's a great library. Recommended.

Aressera said:

Original 262k triangles:

10,000 triangles:

1,000 triangles:

That shows the limitations of triangle removal without texture regeneration. To get down to this level, that wall with arches needs to become one flat surface with the textures projected on it. That's the kind of algorithm I'm looking for, to use on distant buildings.

There are impostor generators, of course. Ones that project onto an octahedron look great on rocks and maybe OK on trees, but terrible on buildings. That approach distorts the verticals and horizontals. The goal for impostors is to enclose the model with as simple geometry as possible, then project the rendered image onto it.

• frob said:
  During asset cooking meshes will have their hierarchical data generated,

That's the problem.

What I'm trying to do is juice up Second Life with a new client. I have one working, using Rust and Vulkan. But handling the big world remains a problem. So I'm working on LOD generation and impostors. Constraints:

• I'm stuck with the existing content, but can do some optimization on it.
• There's very little instancing. That's a consequence of user-created content where you pay for models.
• Scenes are big and complicated. In a city scene, you can go into most of the buildings, walk upstairs, and maybe rent an apartment and move in. I think GTA V has about 40 interiors.
• Everything is modifiable live. There's no “asset cooking” at a global level.
• Long sight lines are common. There's no art direction to prevent that.
• Windows are common. You can see into buildings, and often see in one side and out the other. Occlusion culling does not help much.

Basically, what I want to do is look at large areas of existing content and impostor them, so that viewed from 100m to many kilometers, they look reasonable. Think Google Earth and Microsoft Flight Simulator - good textures on minimized geometry.

(Why Second Life? It's the metaverse that works. It's old and clunky, but everybody else who tried to make a big seamless world with user content did much worse.)

Nagle said:
That shows the limitations of triangle removal without texture regeneration.

I would say the bigger issue here is entire walls missing.

But anyway, the thing we may not like to realize is: The ‘bad’ reduction results are not really bad. We just need to display the model small enough so it's still good enough.
Which means your own results are not bad either, but you switch to this level of reduction too early, while still being much too close to the model. (probably only to show us the bad quality ofc.)

I mean, lod reduction only goes so far. And that's why they often still do the lods manually, e.g. for GTA5 iirc.
It's also why Nanite brags about ‘subpixel detail’ - they can't lod down as early as they would like to.

Here's how GTA V actually does it.

This is from a low poly model of distant terrain someone pulled out of the innards of GTA V. These models are believed to be hand-created. That's what you're really seeing in the distance of GTA V. After lighting, water, and a bit of haze, it looks really good.

I had a go at making models like that automatically using Open Drone Map. That takes in a large number of images and tries to build a 3D model from them using photogrammetry techniques. ODM kept crashing for me, and someone found the problem - the sky dome with clouds is infinitely far away, which confuses the geometric reconstruction and causes the code to loop generating huge triangles. I need to try that again, with an all white or all black sky. ODM is rather brittle; if it doesn't like the data, it crashes, or tries to use infinite memory. It's a wrapper around parts of OpenCV. This looks like it can work but needs considerable effort.

So for the time being, I just took the terrain height map and turned it into large area impostors.

These are some large-area impostors of existing regions in Second Life. They're all the same terrain, at decreasing resolutions. The nearest one is a 256m x 256m area. The middle one is four such areas, including the first one. Notice the identical island. It's four times as big but contains the same number of triangles and texture pixels. The rear one, which you can barely see, represents 64x the area of the nearest area, with, again, the same number of triangles. This scales nicely, all the way up to impostors that represent areas 100km on a side. Overhead is quite low. Each of those is 2048 triangles, regardless of area covered.

This is just built from the height field and a straight-down image, so it's not really very good. Sailors and flyers can see where they are going. I now have the client able to display these, instead of the current infinite water beyond about 200m, which is an improvement. The client has a slippy map algorithm to use the tiles appropriate to the distance.

So I've got a proof of concept, but not good quality yet.

The GTA image is good to observe some properties regarding the 'human factor', if only to help with lowering expectations.

Notice the human factor is two fold.

First regarding optimization objectives, meaning with automation we keep this simple, e.g. setting maximum edge length or divergence from curvature. We use a very small set of values to tweak those objectives.
But the human artist chooses from a wide variety of objectives. He intuitively decides to use much more polygons for the small cranes than for the larger buildings for example.

Second (and that's the bigger problem) regarding the content itself. It makes a huge difference if the original model represents something natural, e.g. smooth bumpy terrain or a creature (easy to reduce), or if it is something human made, e.g. cars or architecture, basically CAD models. For the latter the model itself usually is already reduced to minimum complexity, and reducing it even further quickly fails to reproduce it's appearance. The model also likely has special properties, e.g. parallel planes in architecture walls. A reduction algorithm only analyses local curvature and can not see such global relationships. But the human artist intuitively uses perfect boxes to approximate the buildings, preserving angles which are important for perception to identify it as manufactured, human made stuff.
The third category i see is plants. They grow in a way to minimize volume but maximizing surface area, which generates almost infinite details. This is already hard to model within given poly count constraints, leading to hacks such as alpha textures to approximate details.

So, considering those are all very different things where we are not even happy with modeling all of them with the same textured triangle primitives, we can not really expect general automated reduction good for all of them either. Even if we as humans tend to assume ‘there should be a way’.
I think it is still possible to do better, but only by focusing on special cases, e.g. making a reduction tool which works well primarily for architecture. (I remember some siggraph paper about that which was really good, but i would not find it again.)

Maybe AI can help in the future. But following progress on AI generated 3D models, the better ones look like remeshing. So i guess they generate in 3D volumes similar to 2D image generation, and then remesh the volume. It does not look like there is already AI capable to work with meshes directly. But idk and that's just a guess. However, currently it seems artists can still keep their job for some more time… : |

I spent a lot of thought on how to generate procedural terrain that is LOD friendly, where LODs don't need a full terrain generation step and then simplify the geometry, but can run a simplified generation. Normal heightmap generation like layered perlin noise wouldn't work, because you still need all octaves to properly capture where exactly major outlines are located, such as mountain peaks. And diamon-square subdivsion also doesn't work well, for the same reason. You don't want to move the location of major landmarks between LODs. That's why I came up with a system that generates the exact position of mountain peaks first, so even a simplified version will always have the peak at the same position as the high quality versions. The next layers then add details that are one cognitive order of magnitude below mountain peaks, and so on. So instead of going by mathematical orders of magnitude for layered noise functions, it uses layered functions that each add semantic / cognitive detail, such as smaller hills and valleys. Sounds weird but basically I break down the terrain generation into the features you as a human would be able to point out and name, such as hills, rivers, roads, bridges, cliffs, mountain peaks and valleys, mountain ridges, etc. And then it fills in the detail of those semantic things later on. This allows for much better LODs which only require the first 1-2 layers of the generator, while the next 5 or so layers would only get used for closer observation, without missing out on important features.

Such a generator and LOD system requires lots of codified domain-specific knowledge, though, so it's not applicable to generic geometry. But IMO, if you want to go supper efficient in any area, you need domain-specific knowledge in the code and data model anyway. But such workflows don't work properly with generic asset creation tools, so you might have to write your own 3D modeling tool or a way to tag models made in blender, etc..

So it's a matter of whether you want to put in the effort to write a semantically aware, specialised system, or whether you want to stick with generic data formats and algorithms that have no semantic annotations and therefore no domain-specific knowledge they can leverage.

RmbRT said:
Normal heightmap generation like layered perlin noise wouldn't work, because you still need all octaves to properly capture where exactly major outlines are located, such as mountain peaks.

This can work if your noise layers have a contribution like: amplitude = wavelength * k. (With k being a constant.)
If so, the high frequencies with a short wavelength also have a small height displacement, so you do not need to know them when modeling a level of lower detail than the wavelength.
The basic example is fbm noise.

Just said fyi. fbm and the likes is not that interesting. Your idea of working from predefined peaks is probably better.
fbm may be useful still to add more random detail if needed.

That's how basically all terrain noise functions work, though. You take multiple octaves of noise, each one at half the intensity of the previous one, or some other diminishing factor, and also increased frequency for lower octaves. But if you look at the geometric series (sum of ½^n), you get this problem here:

You'd need at least 3 octaves (yellow) to even somewhat properly locate the peak, even in cases where you don't care about the rest of the shape of the mountain much (so the 4x terrain density would be overkill, especially in 2D where it's 16x). And then you'd still need to run a local search to find the peak. So anything that needs to know where the peak of a mountain is would have to run a local search. In the game I envision, the location of landmarks would be important on a systemic level. Also stuff like rivers. Minecraft can generate rivers on a per-chunk basis, but if you told it to find all rivers in a region and their exact path, it would have to run an exhaustive search over all fully generated terrain in that region. If I start out with the peak and then shape the mountain based on that, I always trivially know the peak.

Of course that part about identifying geometric features for game systems purposes is probably not important for general LODs, but LODs should still preserve (especially cognitively) prominent marks of a shape.

P.S.: And yeah, I would use some more generic noise for just general bumpiness etc. of a piece of terrain. I think that's where noise shines: variation on a scale that is cognitively irrelevant. For anything on a scale that is cognitively relevant, semantic intent should at least exist in the code, even if not system/mechanics-wise in the final output. But chaotic/unguided noise by itself is not good enough for macrostructures. There need to be rules that guide shapes IMO.

So for generic LODs, I would probably want something that lets you “paint” a model, tagging it into separate clusters with that paint, and assigning properties to those. Like an ornamental chair, you paint the ornaments differently maybe from the flatter areas, and allow different LOD strategies that are optimised for different aspects. That way, you still automate the individual vertices, but you get lots of control.

RmbRT said:
layered perlin noise

…made a quick illustration for fbm, where you can disable higher frequencies by selecting the function of a certain octave, to see how the changes become increasingly insignificant with each new octave. And you also see the lowest frequency is still a good approximation of the final detailed result.

https://graphtoy.com/?f1(x,t)=noise(x)&v1=true&f2(x,t)=f1(x)%20+%20noise(x*2)/2&v2=false&f3(x,t)=f2(x)%20+%20noise(x*4)/4&v3=false&f4(x,t)=f3(x)%20+%20noise(x*8)/8&v4=false&f5(x,t)=f4(x)%20+%20noise(x*16)/16&v5=true&f6(x,t)=&v6=false&grid=1&coords=-0.17693282099096763,-2.0565117550509617,4.212999144689288

it really depends. The blue peak in the valley of the 4th big chunk is as prominent as the orange peak right next to it, and the orange peak after that actually has a valley in blue instead.

Peaks and valleys stand out and are more important to get right than the side slope of a mountain. Especially the amplitude and the position of the peak/valley needs to be replicated properly. So even if the horizontal location of the peak or valley sometimes accurate, the amplitude differs wildly (up to 100% error in the second orange peak).

Another part where precision matters is if you have coast lines. You want to accurately represent them without generating a fully detailed heightmap for a huge area. Yet the coastline is basically a threshold function on the finest level of detail. So that's also something that will be extremely hard to identify properly in a LOD. You don't want to have a map that doesn't show entire islands, or shows islands where there aren't any, just because you didn't use the full LOD. So a type of generation process that can define the shape of coastlines beforehand, and then fill in the island, would be extremely valuable for that.

Are you color blind? (It's purple, not blue. Which to my eyes is the most difficult region for identifying hue.)

But yeah, in some spots the difference of all higher frequencies can sum up to the worst case. But the error is still bound to be less than the lowest freq. amplitude, which is a very useful property.

Though, if i try to have more flat terrain but only few peaks, making the noise function more irregular and less smooth, the error can become too big:

(EDIT: It would be this better to the exponential warp on the final result, not on the noise function)

Procedural modelling based on rng is hard not only because randomness is boring, but also because it always tames your creativity in one way or another.

JoeJ said:
Are you color blind? (It's purple, not blue. Which to my eyes is the most difficult region for identifying hue.)

a very blue-ish purple, yeah. Combine that with the contrast to the background and the thin line, looks almost indistinguishable from blue. I don't think I'm red/green blind, though. Maybe there are multiple kinds of colour sight impairment.

JoeJ said:
Procedural modelling based on rng is hard not only because randomness is boring, but also because it always tames your creativity in one way or another.

That's why I want to put as much creativity and design direction into the generator as possible, adding many manually inserted cases, etc.. It won't be the same as fully handcrafted worlds, of course, but if I manage to properly leverage the fact that I have a big world in return for that, I think it will be a worthwhile trade-off. I haven't really seen an open world that properly leveraged its scale, though. All they achieved so far with open worlds was stretch the content thin, and insert lots of boring filler. I would probably discourage 95% of indie studios from making an open world game, especially if procedurally generated.

RmbRT said:
a very blue-ish purple, yeah.

Ah ok. Just remembered somebody on the form was color blind, but not who it was.
It's a matter of communicating hue with terms then. People just disagree on what's purple / blue / pink all the time.

RmbRT said:
I think it will be a worthwhile trade-off.

Yeah, regarding terrain procedural beats manual modeling anyway, which we can utilize.

Architecture is the problem i don't know how to solve yet. I do not like the modelling using instanced modular pieces as seen in current games. I want unique levels, like back then in Quake. And i would sacrifice details for uniqueness if needed.
But even modeling low poly Quake levels is lots of work manually, and idk how to do it procedurally at all.
I'd love to implement CAD in the world editor, but can't afford spending the time needed to figure this out.

JoeJ said:

So, considering those are all very different things where we are not even happy with modeling all of them with the same textured triangle primitives, we can not really expect general automated reduction good for all of them either.

There's an objective metric. Take pictures of different LODs from the same viewpoint. Rescale them to the same size. Apply some blur and haze to unsharp the edges slightly. Compare the images. That's how I check out those island models above. But the LOD builder can't handle buildings or trees yet.

RmbRT said:
I spent a lot of thought on how to generate procedural terrain that is LOD friendly, where LODs don't need a full terrain generation step and then simplify the geometry, but can run a simplified generation.

That's a somewhat different problem. I have existing terrain; I'm not generating terrain. (In Second Life, parcel owners can edit their terrain a bit, so they can create usable building lots in hilly terrain. There's a live “bulldozer tool”).

Terrain isn't the problem, anyway. Buildings and vegetation are the problem.