Sound Sparks Motion: Audio and Text Tuning for Video Editing

The paper proposes Sound Sparks Motion, a test-time framework for motion editing in audio-visual video generation models. Instead of fine-tuning the model, it adjusts two small inputs: an audio latent extracted from the source video and a residual perturbation to the text-conditioning. The authors say this helps models produce localized actions and state transitions that prompt-only control often misses.

Why developers should care: this is a lightweight way to steer generative video without retraining, which matters if you're building editing tools, prototyping AI-assisted cinematics, or trying to preserve existing content while changing motion. The method uses a vision-language model to judge whether the intended motion appears, then regularization and temporal/perceptual constraints to keep the result stable. The paper was submitted on 14 May 2026, and the authors report that the learned latent controls can transfer across videos, suggesting reusable motion directions rather than...