"Just Like This": Manner Deixis at the Graphics Interface

arXiv cs.GR details a conversational 3D authoring technique that turns demonstrations into part of the input loop, letting users specify motion by saying and showing “just like this.” The system combines speech, pointing, and spatiotemporal hand movement to infer a mechanism’s axis, pivot, range, and pace, then renders an animated preview so the interpretation can be checked and revised.

For game developers, the interesting bit is not just the interaction trick but the authoring model: behavior can be sketched quickly, inspected immediately, and refined without forcing everything through a rigid parameter UI. That could be especially useful for designers and technical artists building interactive props, doors, levers, gadgets, or other motion-driven objects where intent is easier to demonstrate than to numerically describe.

The paper reports a 12-participant study comparing three input configurations. The demonstration-assisted version reached 89% participant-declared completion versus 47% with speech alone, and it posted the strongest observed match rates on six categorical accuracy measures while tying on two. Nine participants preferred it, which is a decent signal that the interaction may scale beyond a novelty demo.

The broader takeaway is that authoring tools may benefit from treating motion as a conversation rather than a form. The system can also ask for another demonstration when intent is ambiguous, which points toward a practical hybrid between natural language interfaces and direct manipulation for content creation.