Table of Contents

1. Introduction
2. Overview
3. Installation
4. Tutorial
5. Academical details
6. Contact

1. Introduction

This project calibrates multiple cameras using a planar calibration checkerboard. The pipeline is comprised of four main steps: (1) detect checkerboard corners, (2) remove outlier corners from the previous step (using VAE, variational auto-encoder), (3) estimate initial camera parameters and world (checkerboard) points, (4) refine the initial estimates (using bundle adjustment), then optionally (5) analyze the calibration result.

Left: 16 synchronized cameras. Right: Bundle adjustment in action.

• This calibration method has been used for "Capturing Detailed Deformations of Moving Human Bodies". Chen He, Hyojoon Park, Kutay Macit, and Ladislav Kavan. SIGGRAPH 2021.
• This project is focused on being simple and scalable, applicable to different calibration checkerboards and different number of cameras.
• The steps (1)-(3)/(5) are implemented using python 3 and the step (4) using C/C++.
• On the example dataset provided (./example_data/), this project achieves mean reprojection error 0.08 pixels with standard deviation 0.05.

2. Overview

The code execution follows the pipeline below:

(Input) (0). Multi-view images: The input is a set of images capturing a freely moving checkerboard.

(1). Corner detection: Checkerboard corners are detected and localized with sub-pixel precision using OpenCV.

(2). VAE outlier detector: Outlier corners are identified using VAE (variational auto-encoder), and corresponding images are discarded. As expained here (Section 3), open-source corner detectors (e.g., OpenCV, MATLAB, etc.) are known to give incorrect or inaccurate (i.e., outlier) corners which may deteriorate the accuracy of camera caibration. Therefore, we identify and remove such outliers by tranining a VAE outlier detector. Its effectiveness is explored more here (Section 4).

(3). Initial camera calibration: Initial camera parameters and frame-wise checkerboard poses are first estimated from a subset of images using OpenCV.

(4). Bundle adjustment: The estimated camera parameters and frame-wise checkerboard poses are further refined using bundle adjustment.

(Optional) (5). Analyze the calibration result: Computes reprojection errors and render corresponding histogram, reprojected images, etc. for analysis.

3. Installation

(1) Download the project. In a command prompt (Windows) or terminal (Linux):

git clone https://github.com/hjoonpark/MultiCamCalib.git

(2) Install Ceres

• Click here for Windows
• Click here for Linux (coming)

(3) Build the bundle adjustment project (C/C++)

• Click here

(4) Create Anaconda environment for Python

1. Windows

   Assuming Anaconda is already installed, navigate to ./MultiCamCalib/ where environment.yml exists. Then in a Anaconda Powershell Prompt:

    conda env create --file environment.yml
   

2. Linux (coming...)

(5) Run the codes → Follow this tutorial!

4. Tutorial

Follow this quick start tutorial (a step-by-step example)

5. Academical details

Click here

6. Contact

For questions, please contact Hyojoon Park: hjoonpark.us@gmail.com