Luce: Relightable Gaussians for 3D Asset Generation

Luce is a new 3D asset generation system designed for the part of the pipeline that usually breaks first: turning one image into something that can actually be lit, shaded, and dropped into a real renderer. The core idea is a voxelized multimodal Gaussian cloud with separate primitives for geometry and PBR material signals, so the output can carry albedo, metallic-roughness, and surface normals instead of just a pretty view-dependent shell.

A variational autoencoder compresses that representation into a shared material-aware latent space, and a rectified-flow transformer generates the latent from a single image. The image conditioning comes from multi-layer features of a pretrained encoder, which helps preserve both broad semantics and small details. That matters for production-facing assets, where text, logos, and inscriptions often get mangled by generative systems.

The decoded output can become relightable PBR Gaussians, and it can also produce an optional textured mesh with a tangent-space normal map. On Toys4K, Luce is reported to improve FID by 28% over the strongest baseline, and on a benchmark of AI-generated images it raises CLIP alignment to 0.8519 versus 0.8299. For teams experimenting with AI-assisted asset creation, the practical takeaway is that the focus is shifting from view synthesis toward assets that survive real lighting, material workflows, and downstream engine integration.