SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

SeMoCo is a new motion codec aimed at language-conditioned human-motion generation, and its core idea is to stop treating reconstruction as the only thing that matters. Instead of forcing action meaning and fine-grained movement into the same token hierarchy, each token carries one semantic token plus a residual sequence for kinematic detail. That gives the model a cleaner separation between what the motion means and how it moves.

The generator built around SeMoCo uses a dual-axis design: it models semantic progression over time, then autoregressively refines the residual entries. In practical terms, that should make motion generation more controllable for downstream pipelines that need both intent and fidelity, whether the target is gameplay animation, virtual production, or research tooling.

The work also introduces Ω-MotionVerse, a large multi-source human-motion dataset unified under the SOMA representation. That matters because motion-language systems tend to be bottlenecked by inconsistent data formats and limited coverage; a broader, standardized corpus can improve both training stability and evaluation.

Across the reported comparisons, SeMoCo posts the best reconstruction accuracy among the compared codecs, while also showing strong text-to-motion results. For developers working on animation synthesis, character AI, or motion editing tools, the interesting takeaway is the architectural shift: semantic structure is being treated as a first-class signal rather than an incidental byproduct of compression.