Independent Samples, Correlated Variance A Learnable Cross-View Cue in Path-Traced Stereo Data

arXiv cs.GR details a stereo-rendering quirk that can leak a learnable cue into disparity training. The paper argues that path-traced synthetic stereo is not just carrying independent Monte Carlo noise per eye; once aligned by ground-truth disparity, the per-pixel variance fields become strongly correlated.

Across 20 indoor scenes, warped variance correlation lands at 0.754 ± 0.016 versus 0.360 before alignment, and the effect reproduces on a second renderer with different sampling at 0.743 ± 0.044. A seed-count analysis pushes the estimated population value near 0.85, suggesting the measured numbers are conservative rather than inflated.

The practical takeaway for graphics and ML teams is that stereo networks may be learning from a renderer-specific shortcut, not just image content. A capacity-limited siamese probe using only variance-field patches reached 78.8% two-alternative forced-choice accuracy on held-out scenes, then dropped to chance when alignment was broken. That makes the cue especially relevant for synthetic-data pipelines, evaluation design, and any attempt to close the sim-to-real gap.

The authors also note that the effect is field-level rather than sample-level: individual residuals at the warp correspondence are essentially uncorrelated, while the residual envelope still correlates. In other words, the signal lives in the rendered variance structure itself, which is specific to synthetic data and unlikely to transfer cleanly to real sensors.