Per-Stroke Temporal Control for Text-to-Motion via Action Units and Action-Detection Guidance

Text-to-motion systems have gotten better at generating the right kind of action, but they still struggle with choreography-level timing: asking for four alternating punches often produces motion that reads correctly yet lands poorly. A new approach tackles that gap by making each stroke an explicit conditioning signal instead of relying on a single prompt to carry all the timing intent.

The core idea is Action Units, or AUs. Each AU packages the body track, action class, time window, and impact timing for one stroke, giving the model a structured way to understand not just what happens, but when it happens. That matters for game animation workflows where hit timing, anticipation, and spacing are often more important than broad motion category.

To connect those AUs to an existing text-to-motion backbone, the system uses a lightweight gated adapter with two streams: per-stroke tokens and a per-frame phase channel. At inference, residual timing errors are corrected with training-free classifier gradients from a frozen frame-level detector, which is a practical detail for teams looking to improve control without retraining a full stack.

Evaluation is centered on StrokeBench, a benchmark built around prompts that specify count, ordering, track, and core-frame placement, along with an audited stroke corpus. The method improves correctly placed single strokes over the strongest prior interface while keeping motion quality competitive across text-, interval-, and frame-level baselines. The prompted core frame also becomes another controllable axis, which could be useful for...