I am using Sascha Willems' code ‘raytracingshadows’:

https://github.com/SaschaWillems/Vulkan/tree/master/examples/raytracingshadows​

https://github.com/SaschaWillems/Vulkan/tree/master/data/shaders/glsl/raytracingshadows​

Using the reflectionscene.glft file that I downloaded using download_assets.py, it becomes immediately obvious that, for self-shadowed triangles, the shadow detail is based on the geometry detail of the mesh. This is why you get big squares using this scene (the spheres and teapot are basically quad-based).

I found some answers on reddit, and so I changed the closesthit shader to the following:

...

vec3 lightVector = normalize(ubo.lightPos.xyz);
float dot_product = max(dot(lightVector, normal), 0.0);
hitValue = v0.color.rgb * dot_product + vec3(0.2, 0.2, 0.2);

// Shadow casting
float tmin = 0.001;
float tmax = 10000.0;

vec3 origin = gl_WorldRayOriginEXT + gl_WorldRayDirectionEXT * gl_HitTEXT;
vec3 biased_origin = origin + normal * 0.01;

shadowed = true;  
// Trace shadow ray and offset indices to match shadow hit/miss shader group indices
traceRayEXT(topLevelAS, gl_RayFlagsTerminateOnFirstHitEXT | gl_RayFlagsOpaqueEXT | gl_RayFlagsSkipClosestHitShaderEXT, 0xFF, 0, 0, 1, biased_origin, tmin, lightVector, tmax, 2);
	
if (shadowed) 
{
	hitValue *= 0.3;
}

...

.It looks moderately better, but not perfect:

Any ideas on how to really fix the problem? I was expecting pixel-perfect shadows like on the checkerboard, but no.

In the age of Nanite, does this even matter?

All I know is that it’s not my fault that it looks bad simply because I can detect the edge of the shadow. I did not experience this using shadow maps.

And for the record, I am immensely grateful for the codes by the code author.

Yes this lighting is temporary. multiple lights are in the works. Below is a shot of multiple omnidirectional lights.

This is what shadows looks like using a fractal as an example:

OK, so I perfected the method.

I added the code:

if(dot(normal, lightVector) < 0.0)
	shadowed = true;

and so, the whole shader is:

#version 460
#extension GL_EXT_ray_tracing : require
#extension GL_EXT_nonuniform_qualifier : enable

layout(location = 0) rayPayloadInEXT vec3 hitValue;
layout(location = 2) rayPayloadEXT bool shadowed;
hitAttributeEXT vec2 attribs;

layout(binding = 0, set = 0) uniform accelerationStructureEXT topLevelAS;
layout(binding = 2, set = 0) uniform UBO 
{
	mat4 viewInverse;
	mat4 projInverse;
	vec4 lightPos;
	int vertexSize;
} ubo;
layout(binding = 3, set = 0) buffer Vertices { vec4 v[]; } vertices;
layout(binding = 4, set = 0) buffer Indices { uint i[]; } indices;

struct Vertex
{
  vec3 pos;
  vec3 normal;
  vec2 uv;
  vec4 color;
  vec4 _pad0;
  vec4 _pad1;
 };

Vertex unpack(uint index)
{
	// Unpack the vertices from the SSBO using the glTF vertex structure
	// The multiplier is the size of the vertex divided by four float components (=16 bytes)
	const int m = ubo.vertexSize / 16;

	vec4 d0 = vertices.v[m * index + 0];
	vec4 d1 = vertices.v[m * index + 1];
	vec4 d2 = vertices.v[m * index + 2];

	Vertex v;
	v.pos = d0.xyz;
	v.normal = vec3(d0.w, d1.x, d1.y);
	v.uv = vec2(d1.z, d1.w);
	v.color = vec4(d2.x, d2.y, d2.z, 1.0);

	return v;
}

void main()
{
	ivec3 index = ivec3(indices.i[3 * gl_PrimitiveID], indices.i[3 * gl_PrimitiveID + 1], indices.i[3 * gl_PrimitiveID + 2]);

	Vertex v0 = unpack(index.x);
	Vertex v1 = unpack(index.y);
	Vertex v2 = unpack(index.z);

	// Interpolate normal
	const vec3 barycentricCoords = vec3(1.0f - attribs.x - attribs.y, attribs.x, attribs.y);
	vec3 normal = normalize(v0.normal * barycentricCoords.x + v1.normal * barycentricCoords.y + v2.normal * barycentricCoords.z);


	// Basic lighting
	vec3 lightVector = normalize(ubo.lightPos.xyz);
	float dot_product = max(dot(lightVector, normal), 0.0);
	hitValue = v0.color.rgb * dot_product + vec3(0.4, 0.4, 0.4);

	// Shadow casting
	float tmin = 0.001;
	float tmax = 10000.0;

	vec3 origin = gl_WorldRayOriginEXT + gl_WorldRayDirectionEXT * gl_HitTEXT;
	vec3 biased_origin = origin + normal * 0.01;

	shadowed = true;
	// Trace shadow ray and offset indices to match shadow hit/miss shader group indices
	traceRayEXT(topLevelAS, gl_RayFlagsTerminateOnFirstHitEXT | gl_RayFlagsOpaqueEXT | gl_RayFlagsSkipClosestHitShaderEXT, 0xFF, 0, 0, 1, biased_origin, tmin, lightVector, tmax, 2);
	
	if(dot(normal, lightVector) < 0.0)
		shadowed = true;
	
	if (shadowed) {
		hitValue *= 0.3;
	}
}

You're basically correct. Here are the default shaders:

https://github.com/SaschaWillems/Vulkan/tree/master/data/shaders/glsl/raytracingshadows