Hello,

so this is a wild one. While refactoring my renderer-graph, I stumbled upon an issue with one of my postprocessing-effects. After spending an afternoon trying to see which change caused it, I noticed… it didn't. It even happens with a commit thats way older. I just never noticed since I rarely see that effect while working in the editor, and the effects flickering only happens in very specific circumstances. Full disclaimer, I already know multiple workarounds, but I just want to know what the actual fuck is going on here, since its… odd, to say the least. Anyway, here is the setup:

I have a postprocessing-effect for a 2d-game, to execute a transition where a small rotating box closes in on a location, with black on the outside. Regardless of if I explained that understandably, the setup is this: I have one offscreen-render target where the whole scene is rendered, I feed that as input to the shader and bind another render-target as output, which is then used as the new render-target for the following passes. Finally, in the editor, the last render-target is then composited into the backbuffer, using scissor-rect and a prevesering flip-model to only render those parts that changed. The effect would look like this (frozen; with the bug):

https://gyazo.com/5fe3f354e5d2358a5afe2cccf341b4df​

Now as you can see, the whole rendering flickers. It flickers way slower on the capture then it does in the real world due to FPS-differences. In reality, it also flickers way more often when the mouse is moved, even if offscreen. And here's the deal: The flickering can and should not exist, unless some form of UB or driver-bug is being triggered. You might think, well, its probably due to flipping the render-targets, it uses the wrong one each frame but nope, the actual setup to what is bound to each pass is actually fine, I triple checked with Visual Studio Graphics debugger and all - everything is setup as intended. Speaking of, here is where the fun begins. Let me show you a few of the render-steps as captured with the graphics debugger.

1.) The actual post-processing pass. This is looking fine, always, it renders the exact effect you see in the gif:

2.) Now is where the fuckery begins. This next draw composites the render-target onto the backbuffer:

As you can see, its still looking fine, however this might not always be the case. Randomly, this pass might actually appear to draw the state of the render-target without the transition-effect, even though the correct render-target is bound and when inspecting it at this frame, it does contain the correct state as shown after the last call. But, in this case it seems to work so far, so lets look at the following draw.

3.) This is the draw directly after, and it just draws some text-overlay that overlaps the area of the UI that had to be refreshed:

Uh… ok? Now the effect is gone. See down below in the pipeline-stages, its not like unmodified image is actually drawn. No, its just that the text is drawn and now the debugger somehow forgots that the post-processing has been composited.

4.) The last call, present, is not interesting in this example, however there is another possiblity, as mentioned in 2): In case that the original composite doesn't contain the transition, call 3) might actually make it appear (so invert to what its now), however, the Present will then have the image without it again.

----------------------------------------------------

So yeah, you see why I think thats weird? The debugger seems to be forgetting what it is supposed to actually draw each step, and that also seems to be the case for the actual GPU, which probably results in the blinking. Because unless the GPU somehow eigther drops the postprocessing-pass for one frame or changes the order of draw-calls internally to composite the image one step earlier, there is no way that the final editor-frame would contain the rendering without the transition. But wait, it gets better!

I haven't shown the actual shader yet. Here it is:

#pragma pack_matrix( row_major )
Texture2D Scene: register(t0);

sampler PointWrap : register(s0);
float2 rotatePoint(float2 vPoint, float2 vPivot, float angle)
{
	float s = sin(angle);
	float c = cos(angle);
	vPoint -= vPivot;
	float xnew = vPoint.x * c - vPoint.y * s;
	float ynew = vPoint.x * s + vPoint.y * c;
	return float2(xnew + vPivot.x, ynew + vPivot.y);
}

struct VERTEX_IN
{
	float2 vPos: SV_POSITION0;
	float2 vTex0: TEXCOORD0;
};

struct VERTEX_OUT
{
	float4 vPos: SV_POSITION0;
	float2 vTex0: TEXCOORD0;
};

VERTEX_OUT mainVS(VERTEX_IN In, uint VertexId : SV_VertexID, uint InstanceId : SV_InstanceID)
{
	VERTEX_OUT Out;
	Out.vPos = float4(In.vPos, 0.5, 1.0);
	Out.vTex0 = In.vTex0;
	return Out;
}

struct PIXEL_OUT
{
	float4 vColor: SV_Target0;
};

PIXEL_OUT mainPS(VERTEX_OUT In)
{
	PIXEL_OUT Out;
	Out.vColor.rgb = Scene.Sample( PointWrap,  In.vTex0).rgb;
	
	float2 vCenter = float2(16, 16);
	float fadeValue = 0.5f;
	float2 vScreen = float2(640, 480);
	
	float2 vSize;
	if(abs(fadeValue) < 0.99f)
	{
		vSize = 750.0f * (1.0f - fadeValue / 0.99f) + 18.0f;
	}
	else
	{
		vSize = 0.0f;
	}
	
	
	float2 vPosition = In.vTex0 * vScreen;
	vPosition = rotatePoint(vPosition, vCenter, -3.14156f *(fadeValue / 0.99f) * 2);
	float2 vDistance = abs(vCenter - vPosition);
	if (vDistance.x >= vSize.x || vDistance.y >= vSize.y)
	{
		Out.vColor.rgb = 0.0f;
	}
	
	return Out;
}

I have hardcoded the values that would normally come from the passes cbuffer, to rule out the possibility of that being the issue. Doesn't change anything. However, there are a few things that does change the behaviour. Long story short, the behaviour only happens when the last branch is present. Simplify modifying it to:

float multiply = (vDistance.x >= vSize.x || vDistance.y >= vSize.y) ? 0.0f : 1.0f;
Out.vColor.rgb *= multiply;

Is enough to get rid of the problem. Anything else, regarding sampling the scene, and all other operations are fine. So its also not an issue just with sampling the previous scene, or outputting it, or what is output to the render-target. Now without dragging on for too much longer, here is a list of things that also affect the behaviour:

• Capturing the offscreen-render target before the composite-step always outputs the correct value, and also makes the flickering disappear entirely (so its also a heisenbug)
• Rendering directly to the backbuffer doesn't produce the issue at all
• Changing the framerate/vsync affects the rate of which the flickering appears.
• The more complex the entire render-process is, the less visible the effect becomes at all. In the original setup, the effect is barely visible at all, its more like the inverse where you won't see it at all except for a few frames blinking in. Its only in this simplified setup (only 1 content-pass and then the postprocessing) where the effect will even stay consistently visible

-------------------------------------------------------------

Sorry, that has been a lot. But its a complicated and really weird issue. Its happening on Windows 11, 160HZ HDR monitor, 4090 RTX with drivers from 24.2.2023. I know you don't have all the information about the rendering-setup, and I couldn't show that in such a post. Just also cannot give you an app to try out since it only happens in the mainframe-editor setup. I'm myself trying to figure out if it happens on another PC, but my only other option is a laptop which is slow af, so I don't have results yet. At least the state from a month ago did not produce the issues on eigther the intel or the nvidia card, but I haven't checked if that would prodce it on this PC.

So my question is pretty much just: Does anyone have any clue what could be going on? Has anybody heard about something like this, or experienced it themselves? If its just some weird driver-bug then I'll just implement one of the workarounds, but I'm wondering if it might be something that I'm doing wrong. Is it possible in DX11 to produce such odd behaviour, somehow? I'm obviously also not getting any warnings about binding PS and render-targets to the same texture,

Well, if you have any clue or guess, let me know.

Ok, I can add some further observations:

With the newest state on my old laptop, its also happening. So its not something specific to my setup. At least, it only happens on the Nvidia-card, and not the Intel-HD one. So that means that its a) something specific to Nvidia-cards and b) was caused by a change I made a while ago.

Now the change that actually seems to cause this was setting the optimization-level of the shaders from “D3DCOMPILE_SKIP_OPTIMIZATION” to “D3DCOMPILE_OPTIMIZATION_LEVEL3” in the editor. Well, specifically any optimization-level other than “D3DCOMPILE_SKIP_OPTIMIZATION” triggers it. Still doesn't nearly answer the question of “how” and “why”, but at least its something, I guess?