EgoPHI: Estimating 3D Hand-Object Contact and Force from Egocentric Vision

arXiv cs.GR details EgoPHI, a method for estimating dense 3D hand-object contact and force from a single egocentric RGB image and object geometry. The work is aimed at understanding not just where hands touch, but how force is distributed across hands and objects in physically meaningful ways.

That matters for developers building VR interactions, first-person animation systems, or any pipeline that needs believable hand-object contact. Instead of stopping at image-space contact maps or planar approximations, EgoPHI predicts per-vertex force on hand and articulated object meshes, which is a more useful target for downstream simulation and interaction modeling.

To get around the lack of scalable force labels, the authors built a physics-based simulation pipeline that augments existing hand-object datasets with dense supervision. They also created two physical objects to capture dense contact and force magnitude, then recorded interactions from eight participants across a range of touch and grasp types.

The paper says EgoPHI improves force estimation on both in-distribution and out-of-distribution benchmarks and generalizes to unseen datasets. For game developers, the practical takeaway is that egocentric hand understanding is getting closer to the kind of physically grounded data needed for convincing mixed-reality hands, haptics research, and animation tools that care about pressure as much as pose.