The Domain Is a Residue: Adapting Self-Supervised Features, Not Generators

arXiv cs.GR details a new take on domain translation for graphics and vision: adapt self-supervised features instead of trying to rewrite the whole image pipeline. The Representation Feature Adapter (RFA) is a 2.9M-parameter network that shifts the “residue” of weather, lighting, and rendering style in DINO feature maps while keeping scene content intact.

The practical pitch is straightforward for teams working on de-fogging, de-raining, night enhancement, or sim-to-real transfer. The encoder and a feature-conditioned decoder stay frozen; only the adapter and its discriminators are trained. That makes the method much lighter than generator-heavy approaches, with roughly 160x fewer trainable parameters than CycleGAN-Turbo and under a fifth of its per-condition training time.

On the benchmark side, the method is reported to outperform CycleGAN-Turbo on fog for both metrics, lead on KID for night, and stay roughly on par for snow, rain, and haze. For sim-to-real, it leads REGEN and HyPER-GAN on both metrics. The tradeoff is familiar: preserving more scene structure can mean leaving more of the source domain behind, and the paper notes that CycleGAN-Turbo keeps more structure on every condition except fog.

For graphics programmers and technical artists, the interesting part is the shift in where adaptation happens. If feature-space residue can be moved reliably, it may be easier to build condition-specific pipelines without retraining large generators or paying the usual stability cost of image-to-image translation.