Broadcasting Support Relations Recursively from Local Dynamics for Object Retrieval in Clutters

Yitong Li*, Ruihai Wu*, Hanran Lu, Chuanruo Ning, Yan Shen, Guanqi Zhan, Hao Dong,
Peking University, Tsinghua University, Oxford University
RSS 2024
*Indicates Equal Contribution
Paper Code arXiv

In this paper, we propose a complete pipeline which can analyse complex cluttered objects scenario and infer the support relations between objects in a recursive methodology, which is called ``broadcasting’’. The video shown above demonstrates the inference process, which is comprised by multi-step broadcast until automatically convergence. Simulation and real world experiments demonstrate that our pipeline help robots to gain deep physics understanding for such complicated scenes and show safe and efficient manipulation performance.

Abstract

In our daily life, cluttered objects are everywhere, from scattered stationery and books cluttering the table to bowls and plates filling the kitchen sink. Retrieving a target object from clutters is an essential while challenging skill for robots, for the difficulty of safely manipulating an object without disturbing others, which requires the robot to plan a manipulation sequence and first move away a few other objects supported by the target object step by step. However, due to the diversity of object configurations (e.g., categories, geometries, locations and poses) and their combinations in clutters, it is difficult for a robot to accurately infer the support relations between objects faraway with various objects in between. In this paper, we study retrieving objects in complicated clutters via a novel method of recursively broadcasting the accurate local dynamics to build a support relation graph of the whole scene, which largely reduces the complexity of the support relation inference and improves the accuracy. Experiments in both simulation and the real world demonstrate the efficiency and effectiveness of our method.

Clutter scenarios in Daily Life

MY ALT TEXT

This figure show a variety of scenarios which contain cluttered objects. Retrieving a specific object in these scenarios requires robots to understand the physics structure and manipulation in a feasible path. Otherwise the supported objects will fall down and cause damage since the fragile material.

Method: Infer Support Relation with Recursive Broadcasting

MY ALT TEXT

This figure illustrates the framework for inferring support relations. Initially, we focus on the target object and its adjacent inferring the support relations between them with particle-based dynamics model, referred to as the “Broadcasting” process. Next, for the objects identified as child nodes of the target object, i.e. supported by the target object, we recursively apply the “Broadcasting” process to these child nodes and continue this process for the subsequent child nodes, and so on. This recursion continues until no new child nodes emerge during the latest inference. This method ensures that all support relations related to the target objects are fully identified.

Method: Dynamics Adjustment in Long-horizon Manipulation

MY ALT TEXT

For such a long-horizon task, the observation will change during manipulation. For the observation before manipulation, there are many occlusions. But during manipulation, the occlusion may be removed and many details will be exposed to the camera. so that we can re-broadcast on the next manipulation object and make a better estimation.

Real World Experiments Demonstrations

Video Presentation

BibTeX

@inproceedings{
        li2024broadcasting,
        title={Broadcasting Support Relations Recursively from Local Dynamics for Object Retrieval in Clutters},
        author={Li, Yitong and Wu, Ruihai and Lu, Haoran and Ning, Chuanruo and Shen, Yan and Zhan, Guanqi and Dong, Hao},
        booktitle={Robotics: Science and Systems},
        year={2024}
        }