This paper addresses the multi-faceted problem of robot grasping, where multiple criteria may conflict and differ in importance. We introduce Grasp Ranking and Criteria Evaluation (GRaCE), a novel approach that employs hierarchical rule-based logic and a rank-preserving utility function to optimize grasps based on various criteria such as stability, kinematic constraints, and goal-oriented functionalities. Additionally, we propose GRaCE-OPT, a hybrid optimization strategy that combines gradient-based and gradient-free methods to effectively navigate the complex, non-convex utility function. Experimental results in both simulated and real-world scenarios show that GRaCE requires fewer samples to achieve comparable or superior performance relative to existing methods. The modular architecture of GRaCE allows for easy customization and adaptation to specific application needs.
<img align="center" alt="GraspFlow" src="figs/main_fig3.png" width="600" height="300" />
This project has a lot of submodules. We recommend to use forked versions of submodules given in this repository. First, create and activate conda environment and install modules given in environment.yaml:
conda env create -f environment.yml
We use Pytorch version of the Graspnet.
- Pointnet2_PyTorch (given as a submodule in this repo) - PointNet++ library for GraspNet's backbone.
- franka_analytical_ik - solves analytical IK for Panda Robot.
We mainly follow same installation as in. However, we also extended it to add additional filtering capabilites. Please install IK submodule and copy generated library to pytorch_6dof-graspnet module. Details are given in this link.
Note: Generally any grasp sampler can be used. In our paper, we have also tested on GPD sampler.
- franka_analytical_ik - solves analytical IK for Panda Robot.
- differentiable-robot-model - differentiable robot model used for E-classifier to calculate FK of the robot.
- Task-Grasp - Classifier for grasps with respect to the intent provided by a user
- JointBERT - Intent classification model for grasp queries from a user
Each of the submodule has own documentation for the setup.
To optimize grasps, go to GoES folder:
cd GoES/GoES
and run:
python refine_isaac_grasps.py --cat <cat> --idx <idx>\
--grasp_folder ../experiments/test \
--sampler graspnet --grasp_space SO3 --device 0 --max_iterations 30 \
--method GraspOptES --classifier <classifier> --experiment_type E_TYPE \
--cfg configs/graspopt_isaac_params.yaml --batch_size 15
For list of parameters for the above function, please check settings.txt file.
Please read README under graspflow-simulator folder.
To cite this work, please use:
TODO