UltraDiff: Differentiable Ray Tracing in Ultrasound for Shape Optimization
arXiv cs.GR details UltraDiff, a differentiable ray-tracing framework that extends physically based optimization from light transport to medical ultrasound. The core idea is to model image formation as a path-space integral gated by travel time, then derive gradients that let the system tune scene parameters against measured echoes.
For developers working in rendering or simulation, the interesting part is the pipeline: UltraDiff is built on Mitsuba 3 and uses a Monte Carlo estimator for both the forward model and its gradients. In practice, that means the same analysis-by-synthesis loop familiar from differentiable rendering can be applied to a very different sensing modality.
The paper demonstrates inverse geometry estimation by starting from a sphere and optimizing an SDF until simulated echoes line up with measurements. It recovers vertebral surfaces from both simulated B-mode sweeps and a real robotic acquisition of a spine phantom, without relying on pre-segmented images.
The broader takeaway is that differentiable path tracing is no longer limited to optics. Even if ultrasound is far from game production, the technique is a useful signal that inverse problems, gradient-based fitting, and physically grounded simulation continue to spread across graphics-adjacent research.
“We extend this paradigm to medical ultrasound”
- what
- UltraDiff is a differentiable ultrasound ray-tracing framework for shape optimization.
- who
- Authors: Felix Duelmer, Magdalena Wysocki, Nassir Navab, and Mohammad Farid Azampour.
- when
- Submitted to arXiv on 6 Oct 2026.
- impact
- Shows gradient-based inverse rendering can fit geometry from ultrasound B-mode data without pre-segmentation.
Research advance; interesting but not directly game-facing
Follow graphics research updates
See relevant stories in your personalized news feed.
Discussion