More Works from Our Lab

Learning Geometric Reasoning Networks for Robot Task and Motion Planning

We present Geometric Reasoning Networks (GRN), a graph neural network-based model for predicting action and grasp feasibility in Task and Motion Planning (TAMP). GRN reduces reliance on geometric planners by incorporating interpretability mechanisms such as inverse kinematics feasibility prediction and grasp obstruction estimation, enabling efficient and explainable planning in complex 3D environments.

The Thirteenth International Conference on Learning Representations (ICLR 2025)
Simultaneous Action and Grasp Feasibility Prediction for Task and Motion Planning through Multi-Task Learning

We propose AGFP-Net, a multi-task neural network that predicts action feasibility and grasp type feasibility, significantly improving task and motion planning (TAMP) by reducing geometric planning time and solving more complex problems.

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)
Learning to Predict Action Feasibility for Task and Motion Planning in 3D Environments

We introduce a novel approach for Task and Motion Planning (TAMP) in 3D environments, combining an efficient 3D scene representation with a deep neural network to predict action feasibility, significantly reducing geometric planning time by up to 90% on complex problems.

2023 IEEE International Conference on Robotics and Automation (ICRA)

Extending Task and Motion Planning with Feasibility Prediction: Towards Multi-Robot Manipulation Planning of Realistic Objects

Smail Ait Bouhsain, Rachid Alami, Thierry Siméon
LAAS-CNRS
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)
Paper Poster Code HAL

Our method addresses the combinatorial challenges of task and motion planning (TAMP) in multi-robot systems by introducing a feasibility-enabled TAMP algorithm. Leveraging advancements in action and grasp feasibility prediction, our approach extends these predictions to mesh-shaped objects and multi-robot settings, enabling the symbolic planner to focus on geometrically feasible solutions. This significantly reduces planning time and enhances the system's ability to solve complex multi-robot manipulation tasks requiring collaboration.

Abstract

The hybrid discrete/continuous nature of task and motion planning (TAMP) results often in a combinatorial explosion. This challenge is even more pronounced in multi-robot TAMP problems due to the increase in dimensionality of the action space. Previous works use action feasibility prediction as a heuristic to accelerate TAMP. However, these methods are limited to box-shaped objects and specific single or dual robot settings. In this paper, we propose a feasibility-enabled multi-robot TAMP algorithm capable of tackling complex multi-robot manipulation problems. Also, we expand on our previous work on action and grasp feasibility prediction [1] by extending its use to mesh-shaped objects. We demonstrate the performance of our method compared to a non feasibility-informed baseline, and show its ability to handle TAMP problems requiring the collaboration of multiple robots..

Proposed Model

Diagram of AGFPNet

The action and grasp feasibility prediction neural network architecture (AGFPNet).

Diagram of AGFPNet's extension to multi-robot settings

A visualization of the proposed extension of AGFPNet to multi-robot settings.

Diagram of AGFPNet's extension to mesh-shaped objects
Diagram of AGFPNet's extension to mesh-shaped objects

A visualization of the proposed extension of grasp representation to mesh-shaped objects.

Benchmarks

Access

Access problem: A single robot has to move a meat can. A number of mesh objects are however blocking access to it, which requires removing all blocking objects to access the wanted one, before returning them to their initial pose.

Sort Benchmark

Sort problem: A dual-robot task where objects must be sorted onto two pre-occupied tables. The challenge involves finding feasible placements on narrow surfaces, with each robot restricted to accessing only one table, and a subset of the objects only.

Clear Benchmark

Clear problem: Two large objects (in red) span over the intersection region of two robots’ workspaces, rendering Pass actions infeasible. The robots have to first clear the intersection region before moving a set of objects from one table to another.

Longswap Benchmark

Longswap problem: Three sequential robots have to collaborate to swap the placements of 4 objects. The intersection regions between robots’ workspaces are, however, partially blocked by a set of obstacles.

Middleman3 Benchmark

Middleman3 problem: Three robots form a triangle such that there is an intersection between every pair of robots' workspaces. One of the intersection regions is blocked by a fixed obstacle, forcing the robots to perform two $Pass$ actions via a middleman robot to move three objects from one counter to the other.

Middleman5 Benchmark

Middleman5 problem: Four symbolic action sequences allow each one of four objects to be moved from a shelf to a table, however three of them are blocked by fixed obstacles. The planner must find the only geometrically feasible sequence of robots exchanges to solve the problem.

Results

Combined Results

Visualizations

Plan execution visualizations for each problem.

Visualization of the solution to the Access Problem

Access Problem

Visualization of the solution to the Sort Problem

Sort Problem

Visualization of the solution to the Clear Problem

Clear Problem

Visualization of the solution to the Long-swap Problem

Long-swap Problem

Visualization of the solution to the Middleman3 Problem

Middleman3 Problem

Visualization of the solution to the Middleman5 Problem

Middleman5 Problem

BibTeX

          
            @inproceedings{ait2024extending,
              title={Extending Task and Motion Planning with Feasibility Prediction: Towards Multi-Robot Manipulation Planning of Realistic Objects},
              author={Ait Bouhsain, Smail and Alami, Rachid and Simeon, Thierry},
              booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
              pages={10318--10325},
              year={2024},
              organization={IEEE}
            }