PART: Learning 3D Part Assembly and Retrieval with Transformers

arXiv cs.GR details PART, a unified transformer framework for 3D part retrieval and assembly. Given a target shape and a library of parts, it selects components and predicts their 6-DoF poses to rebuild the target, turning assembly into a set-prediction problem instead of a fixed-part matching task.

That matters because retrieval-based assembly is much closer to real production workflows than systems that assume a pre-defined kit of pieces. The hard parts here are the combinatorial search space, variable output length, and continuous pose estimation, all of which become painful once the part library grows or the target geometry changes.

The paper pairs retrieval and pose estimation with joint training, then adds a segmentation-enhanced optimization module that exploits the link between part placement and target segmentation. The authors also curated a dataset of more than 80,000 shapes, giving the method a scale that should be useful for benchmarking and follow-on work.

For game developers, the practical angle is obvious: modular asset assembly, scan cleanup, scene reconstruction, and procedural content tools could all benefit from better part selection and placement. The authors say the system generalizes to scene layouts, image targets, and real-world scans, which suggests the technique may be relevant beyond clean CAD-style inputs.