Skip to main content
GameDev.net gamedev.net
🔒 Locked

Depth Based Fog -- How to handle outlines around objects when using a depth based fog post process? Perhaps depth buffer imprecision issue?

Started by enigma_dev Aug 13, 2022 at 2:27 PM 2 replies 8.8k views
Original Post
enigma_dev
enigma_dev

I am implementing a depth based fog in a post process shader/material (ue5 for what it is worth, but I imagine I will have same issue I'd have in OpenGL).
The outline of objects shows a very slight line around the object.
I'm thinking this is because the depth buffer isn't granular enough around the edges of the object. There's a large discrepancy in distance between the object and its background.


On the right, you can see my shader/materia (gray fog)l; there is a whitish outline that flickers; most visible on the right side of the tree.
On the left, you can see a stock environmental height fog from the engine (blue fog), with no outlines.
Unfortunately, I'm not able to see how the engine's fog is implemented. (right)

I'm curious if anyone has encountered anything like this before and know a general approach for solving it?
I'm most familiar with glsl
the current material is essentially:

//pseudocode - as above picture is coming from a material graph
uniform float maxdepth = 30000.0;
uniform float fogcolor = vec3(0.5, 0.5, 0.5);
void main(){
  vec3 scenecolor = …;
  float scenedepth = …;
  float lerp_alpha = clamp(scenedepth/maxdepth, 0, 1);
  outcolor = lerp(scenecolor, fogcolor, lerp_alpha); //when alpha is 1, we get 100% fog color, I might have mixed this ordering up in glsl
}
enigma_dev
enigma_dev

Following up on this thread.
It actually isn't related to depth imprecision like I originally thought.

This is a post processing shader.
The antialiasing is being applied before the post process is being applied.
It is a temporal based antialiasing. (TSR temporal super resolution)
Switching to FXAA or MSAA helps, but appears to not look as good for things like global illumination.
Fortunately, I can specify the material to happen before tone mapping (which appears to also put it before Anti Aliasing)
This fixes the issue, but the fog color does it get tone mapped.

It looks okay to me at the moment though.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.