title

Self-supervised Learning of Reconstructing Deformable Linear Objects under
Single-Frame Occluded View

Department of Mechanical Engineering, Tsinghua University
ICRA2025

Real-world Demo of Occlusions and Self-intersections

Video

Abstract

Deformable linear objects (DLOs), such as ropes, cables, and rods, are common in various scenarios, and accurate occlusion reconstruction of them are crucial for effective robotic manipulation. Previous studies for DLO reconstruction either rely on supervised learning, which is limited by the availability of labeled real-world data, or geometric approaches, which fail to capture global features and often struggle with occlusions and complex shapes. This paper presents a novel DLO occlusion reconstruction framework that integrates self-supervised point cloud completion with traditional techniques like clustering, sorting, and fitting to generate ordered key points. A memory module is proposed to enhance the self-supervised training process by consolidating prototype information, while DLO shape constraints are utilized to improve reconstruction accuracy. Experimental results on both synthetic and real-world datasets demonstrate that our method outperforms state-of-the-art algorithms, particularly in scenarios involving complex occlusions and intricate self-intersections.

Framework

Self-supuervised DLO occlusion Single-Frame Reconstruction Framework

Framework PNG

A. Self-supervised DLO Point cloud completion, B. Ordered key points generation.

Experiments

Simulation Experiments—Synthetic Dataset Generation

The DLO has randomly moving endpoints, while an occlusion cube moves to simulate occlusions.

The RGB image

The Generated Point Cloud

Simulation Experiments—Comparison Results


Label PNG


Case 1

Case 2

Case 3

Case 4

Case 5

Case 6

Real-world Experiments—Training Stage

Simplify data collection by manual manipulation of both ends of DLO under an RGB-D camera.

Manually Collected Scene

The Collected Point Cloud


Real-world Experiments—Inference Stage

Visualization of Occlusions and Self-intersections

Normal Scenario

Complex Scenario

More Single-Frame Details

Contact

Please contact us at wangs23@mails.tsinghua.edu.cn, if you have any question.