EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing

EditStream is a unified video generation and editing framework built around a DiT-based model with task-specific conditioning. Instead of treating each creative workflow as a separate system, it handles text-to-video, image-to-video, video-to-video, editing propagation, reference-guided editing, and camera pose changes inside one pipeline.

The practical hook is speed: EditStream is distilled into a fast, few-step autoregressive model for streaming-style interaction. That matters for game teams and tools developers because interactive iteration is often blocked by latency, not just model quality. A system that can respond quickly enough for live creative workflows is much easier to imagine inside editor tooling, previs, or content prototyping.

To make the unified model usable, the team uses a two-stage distillation approach. Velocity Moment Matching preserves conditional velocity moments at intermediate student states to keep generation quality and motion intact, while autoregressive unrolling teaches the model to survive its own predictions and stay temporally stable over time.

The main pain points it targets are familiar ones: over-saturation, motion degradation, temporal instability, and the complexity of training separate models for separate tasks. For developers building content pipelines, the broader implication is a push toward one controllable video system that can support both generation and editing without forcing a workflow split between tools.