OBJECT DETECTION AND RECOGNITION IN 3D Point Cloud scene

DISCLAIMER: project is still in development stage, I try to keep master branch as updated and functional as possible.

Dependencies

• ROS Kinetic
• PCL 1.8
• Opencv 2
• Python 2.7
• Numpy
• cv2
• TensorFlow
• Keras

Nodes

	|-> tf_emitter
	|-> segmentationNode
	|-> classifierNode
	|-> RViz
	`-> pcd_to_pointcloud

tf_emitter

• publishes world frame for RViz

segmentationNode

• publishes all the segmented clouds to /pcVector

classificationNode

• subscribes /pcVector and classifies every segmented pointcloud using 3DCNN and publishes bounding boxes as MarkerArray

Rviz

• used for visualization

pcd_to_pointcloud

• to publish scenes for testing

Note

/bin - contains .pcd scenes which are broadcasted

Dataset

https://rgbd-dataset.cs.washington.edu/dataset/rgbd-dataset_pcd_ascii/

Directory Structure for DataSet

Dataset_RGBD
	|
	├── apple_1
	|		|── apple_2_4_9.pcd
	|		.
	|		`── apple_2_1_111.pcd
	├── banana_1
	|		|── banana_1_4_6.pcd
	|		.
	|		`── banana_1_4_80.pcd
	├── bowl_1
	|		|── bowl_2_1_9.pcd
	|		.
	|		`── bowl_3_4_119.pcd
	├── calculator_1
	|		|── calculator_2_1_2.pcd
	|		.
	|		`── calculator_3_2_138.pcd
	└── coffee_mug_1
	 		|── coffee_mug_3_4_6.pcd
	 		.
	 		`── coffee_mug_3_4_83.pcd

Training the classifier

rosrun robot_vision trainer.py

To run package

roslaunch robot_vision robot_vision3.launch