More Works from Our Lab

Learning Geometric Reasoning Networks for Robot Task and Motion Planning

We present Geometric Reasoning Networks (GRN), a graph neural network-based model for predicting action and grasp feasibility in Task and Motion Planning (TAMP). GRN reduces reliance on geometric planners by incorporating interpretability mechanisms such as inverse kinematics feasibility prediction and grasp obstruction estimation, enabling efficient and explainable planning in complex 3D environments.

The Thirteenth International Conference on Learning Representations (ICLR 2025)
Extending Task and Motion Planning with Feasibility Prediction: Towards Multi-Robot Manipulation Planning of Realistic Objects

We introduce a multi-robot TAMP algorithm that leverages feasibility prediction to address complex manipulation tasks. Our approach extends previous methods by enabling the handling of mesh-shaped objects and collaborative multi-robot settings, demonstrating significant improvements over baseline methods.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024)
Simultaneous Action and Grasp Feasibility Prediction for Task and Motion Planning through Multi-Task Learning

We propose AGFP-Net, a multi-task neural network that predicts action feasibility and grasp type feasibility, significantly improving task and motion planning (TAMP) by reducing geometric planning time and solving more complex problems.

2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023)

Learning to Predict Action Feasibility for Task and Motion Planning in 3D Environments

Smail Ait Bouhsain, Rachid Alami, Thierry Siméon
LAAS-CNRS
IEEE International Conference on Robotics and Automation (ICRA 2023)
Paper Poster Code HAL

Given the state of a 3D environment, a high-level action, and an object to manipulate, our method predicts the geometric feasibility of the action. These predictions are then used as a heuristic to guide task and motion planning, by prioritizing the most promising actions first. This approach significantly reduces the time spent on geometric planning, enabling the robot to solve manipulation tasks more efficiently in 3D environments.

Abstract

In Task and motion planning (TAMP), symbolic search is combined with continuous geometric planning. A task planner finds an action sequence while a motion planner checks its feasibility and plans the corresponding sequence of motions. However, due to the high combinatorial complexity of discrete search, the number of calls to the geometric planner can be very large. Previous works [1] [2] leverage learning methods to efficiently predict the feasibility of actions, much like humans do, on tabletop scenarios. This way, the time spent on motion planning can be greatly reduced. In this work, we generalize these methods to 3D environments, thus covering the whole workspace of the robot. We propose an efficient method for 3D scene representation, along with a deep neural network capable of predicting the probability of feasibility of an action. We develop a simple TAMP algorithm that integrates the trained classifier, and demonstrate the performance gain of using our approach on multiple problem domains. On complex problems, our method can reduce the time spent on geometric planning by up to 90%.

Proposed Model

Diagram of the proposed neural network AFPNet

A visualization of the proposed neural network architecture used for action feasibility prediction.

Benchmarks

Reorder Benchmark

Reorder problem: Objects initially placed on a two-shelf cupboard must be moved to another cupboard in a different order, testing the algorithm's ability to handle grasp choices and infeasible placements under shelves.

Unpack Benchmark

Unpack problem: Objects initially placed on a tray-like surface must be unpacked and ordered into a cupboard. The challenge lies in selecting feasible grasps due to the proximity of objects and the restricted goal placements under a shelf, which disallow top grasps.

Swap Benchmark

Swap problem: A set of objects are initially placed either on the table or on higher support surfaces, and the goal is to swap their poses. The challenge arises from the fact that the goal poses of objects are already occupied, requiring the use of intermediary placements to achieve the desired configuration.

Results

Results on 2-object problems

Detailed planning time of our algorithm on the 2-object version of the problems with and without using AFP-Net, averaged over 10 runs.

Results on 5-object problems

Detailed planning time of our algorithm on the 5-object version of the problems with and without using AFP-Net, averaged over 10 runs.

Overall Results Comparison

Comparison of the performance of the TAMP algorithm with and without using AFP-Net on the 5-object version of the three problem domains. Results are averaged over 10 runs.

Visualizations

Visualization of the solution to the reorder task

Execution of the reordering task with 5 objects.

Visualization of the solution to the unpack task

Execution of the unpacking task with 5 objects.

Visualization of the solution to the swapping task

Execution of the swapping task with 5 objects.

BibTeX

          
            @inproceedings{ait2023learning,
              title={Learning to predict action feasibility for task and motion planning in 3d environments},
              author={Ait Bouhsain, Smail and Alami, Rachid and Simeon, Thierry},
              booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
              pages={3736--3742},
              year={2023},
              organization={IEEE}
            }