SSFold:Learning to Fold Arbitrary Crumpled
Cloth Using Graph Dynamics from Human Demonstration

Robotic cloth manipulation faces challenges due to complex dynamics and the high dimensionality of configuration spaces. Previous methods have largely focused on isolated smoothing or folding tasks and heavily relied on simulations, often failing to bridge the significant sim-to-real gap in deformable object manipulation.

To overcome these challenges, we propose a two-stream architecture with sequential and spatial pathways, unifying smoothing and folding tasks into a single adaptable policy model that accommodates various cloth types and states. The sequential stream determines cloth pick and place positions, while the spatial stream, leveraging a visible connectivity dynamics model, constructs a visibility connectivity graph from partial point cloud data of self-occluded cloth, thus improving the robot’s perception of the cloth’s current state. To bridge the sim-to-real gap, we utilize a hand tracking detection algorithm to gather and integrate human demonstration data into our novel end-to-end neural network, improving real-world adaptability. Our method, validated on a UR5 robot across four distinct cloth folding tasks, reliably achieves folded states from crumpled initial configurations with success rates of 99%, 99%, 83%, and 67%. It outperforms existing state-of-the-art cloth manipulation techniques and demonstrates strong generalization to unseen cloth with diverse colors, shapes, and stiffness in real-world experiments.