LightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting

LightCrafter reframes video relighting around a proxy render: first generate a PBR version of the input under the desired lighting, then use video diffusion to refine that result into the final output. For game teams, the practical appeal is obvious: the lighting target is baked into the proxy, so the model no longer has to learn environment maps or other illumination concepts directly.

That matters because the two common approaches each have familiar pain points. Inverse-rendering pipelines can be noisy and tend to fall apart on hard lighting effects like global illumination, while pure generative video-to-video methods often struggle with long clips, consistency, and training data requirements. LightCrafter is trying to sit between those extremes by using physically grounded rendering where it helps and diffusion where it fills in the gaps.

To handle the missing pieces, the system post-trains CogVideoX on synthetic video pairs and real-world unpaired footage. The team reports better results than prior state-of-the-art on existing real-world relighting benchmarks, and it also introduces a synthetic benchmark for deeper analysis. The paper is dated July 9, 2026, with a revision on July 15, 2026.

For graphics programmers and technical artists, the interesting part is the hybrid workflow: it suggests a path to controllable relighting without giving up temporal coherence. The authors also plan to release the dataset, benchmark, metrics, and code, which should make it easier to test whether this approach holds up in production-style pipelines.