DATASETS

Objects in the controlled environments :

This table lists the datasets commonly known as BOP: Benchmark 6D Object Pose Estimation, which provide accurate 3D object models and accurate 2D-3D alignment.

You can download all the BOP datasets here and use the toolkit provided by the organizers.

After downloading the data, you can use our code data/BOP/ply2obj.py to convert original .ply files to .obj files, and run data/BOP/create_annotation.py to create a single annotation file for all the scenes in a dataset.

Datasets format can be found here, we use instance id in our annotation to indicate different instances pictured in the same image.

Dataset Sample image Annotation Statistics Reference
Objectron 3D Bounding Box 15K annotated videos and 4M annotated images CVPR 2021
MetaGraspNet Depth + Mask 100,000 RGBD images, 11,000 scenes, and 25 classes of objects ArXiv 2021
GraspNet-1B 6D pose + Depth + Mask 88 models in 190 videos with 97,280 frames CVPR 2020
NOCS 6D pose + Depth + NOCS 6 categories in 300K composited images and 8K real images CVPR 2019
YCBInEOAT 6D pose + RGBD Video 5 models in 9 videos with 7449 frames with moving objects IROS 2020
YCB-Video 6D Pose + Depth + Mask 21 models in 92 videos with 133,827 frames RSS 2018
T-LESS 6D Pose + Depth 30 models in 20 videos with ~49K frames WACV 2017
Doumanoglou 6D Pose + Depth 2 models in 3 videos with 183 frames CVPR 2016
Tejani 6D Pose + Depth 6 models in 6 videos with 2,067 frames ECCV 2014
Occluded-LINEMOD 6D Pose + Depth 8 models in 1,214 frames with 8,992 objects ECCV 2014
LINEMOD 6D pose + Depth for one object 15 models in 15 videos with 18,273 frames ACCV 2012

datasets

3D model datasets :

In order to testify the network generalization ability (tested on images containing unseen 3D models from the training set), the following dataset could be used to generate synthetic training data.

Notice that ABC contains generic and arbitrary industrial CAD models while ShapeNetCore and ModelNet contain common category objects such as cars and chairs.

Dataset Categories Models in total Reference
ABC - 1 million CVPR 2019
ShapeNetCore 55 ~51,300 ArXiv 2015
ModelNet-40 40 12,311 CVPR 2015

Rendering methods :

Differentiable Renderer

Blender Render :

Physical Simulator

PyBullet: a very popular one in the Robotics community.

Others

  • Glumpy: does not support headless rendering (failed on ssh mode)

  • UnrealCV: extension of Unreal Engine 4, helps interact with virtual world and communicate with external program.

  • SyntheticComputerVision: resuming a lot of techniques used to generate synthetic image