arXiv 2409.08273
Hand-Object Interaction Pretraining from Videos
By Himanshu Gaurav Singh, Antonio Loquercio, et al.
Published 2024-09-12
Wiki summary
Explore the paper's summary, context, and related research on Papiers.
We present an approach to learn general robot manipulation priors from 3D hand-object interaction trajectories. We build a framework to use in-the-wild videos to generate sensorimotor robot trajectories. We do so by lifting both the human hand and the manipulated object in a shared 3D space and retargeting human motions to robot actions. Generative modeling on this data gives us a task-agnostic base policy. This pol…