Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter
Chaitanya Mitash, Abdeslam Boularias and Kostas E. Bekris
International Journal of Robotics Research (IJRR) 2019
This webpage provides a complete solution for object pose estimation in clutter. The following components are shared:
- Autonomous data generation to train CNNs for object segmentation (synthetic data)
- Rutgers Extended RGBD dataset (test dataset comprising real pose-labeled scenes)
- Search-based object pose estimation process.
Autonomous data generation to train CNNs for object segmentation
The training dataset is generated by physical simulation of the setup in which the robot operates. The tool we developed for autonomous data generation, labeling, and training is shared below.
Dataset Generation toolbox: https://github.com/cmitash/physim-dataset-generator
Rutgers Extended RGBD dataset
Dataset download link: download
For each scene in the dataset, we share:
- RGB Image
- Depth Image
- Segmentation mask
- camera_pose: pose of the camera in a global frame.
- camera_intrinsics: intrinsic parameters of the camera.
- rest_surface: pose of the resting surface such as a table or shelf bin.
- dependency_order: physical and visual dependency of objects upon each other.
- pose: ground-truth object pose in a global frame.
Examples of scenes in the dataset and results of pose estimation with physics-based reasoning.
Object Pose Estimation
Chaitanya Mitash, Kostas E. Bekris, and Abdeslam Boularias, A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation
In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, 2017
Chaitanya Mitash, Abdeslam Boularias and Kostas E. Bekris, Improving 6D Pose Estimation of Objects in Clutter via Physics-aware Monte Carlo Tree Search
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 2018
Chaitanya Mitash, Kostas E. Bekris, and Abdeslam Boularias
Computer Science Department, Rutgers University, New Brunswick, NJ.