Marc Jones
The image analysis pipeline has been developed in Python using the OpenCV computer vision library [1] and consists of: i) Detection of the calibration window; ii) Image normalization; iii) Prediction of algal density.
Images are converted to greyscale and have a Gaussian blur applied to them before they are subjected to a threshold. The threshold converts images into black and white images, to which a contour finding algorithm is applied. Contours are arranged into a tree like structure, with a contour's parent being the contour which contains it. In order to find the three anchor points of the calibration window the algorithm iterates over the list of contours and finds contours whose areas are in the correct ratios. The ratios which the algorithm looks for are a 9:16 area ratio between a conotur and its parent and a 9:25 area ratio between a contour and its grandparent. Once three anchor points are found, the image is transformed to orient the picture correctly and correct any skewing of the image.
Using the positions of the anchor points, the coloured squares and the transparent window are located in the image and the pixel information for them extracted. To allow the algorithm to run in reasonable time, the number of pixels may be downsampled. To normalize the colours, a Gaussian process model is applied to each colour channel (red, green and blue) separately. Gaussian processes are a probabilistic framework used to model unknown functions, and are used in this study because of the unknown non-linearities involved when normalizing photos taken using different equipment and lighting conditions. They are implemented in the algorithm using the GPy Python library [2]. To parameterize the Gaussian process models, pixel information from the coloured squares and from the black squares within the anchor points are used. Positional as well as RGB values of the pixels are combined to ensure that variation due to the camera and lighting can be taken into account. The Gaussian process models are constructed using a linear kernel combined with a squared exponential kernel. This allows the model to capture the general linear trend while also allowing for non-linearities to be considered by the model. The pixel information from an image as well as from a reference image of the calibration window are used to train the models for each colour channel. Once parameterized, the Gaussian process models are used to normalize the pixel information from the transparent section of the calibration window, which corresponds to the colour of the algae in the bioreactor. Each pixel is normalized by inputting its position in the image as well as its unnormalized RGB values into each Gaussian process model. The output from the Gaussian process models are mean and variance values for each colour channel (Figure 1).
In order to relate the colour of the algal reaction mixture to algal density, the distribution of values for each colour channel needs to be known. For each pixel normalized using the Gaussian process method detailed above, a normalized mean value and a variance value are given as output for each colour channel. Finding the algal reaction mixture colour values for each channel by averaging the normalized pixel mean colour values would not incorporate the variation observed for each pixel. Therefore, to incorporate the variation, the per pixel probability distributions are combined and points are sampled from the combined probability distribution. An overall mean and variance for the algal reaction mixture for each colour channel are then calculated from the sampled values. The colour values for the algal reaction mixture are used as inputs to another Gaussian process model to allow for prediction of the algal density measurements (Figure 2).
- Bradski, G., OpenCV Library, Dr. Dobb's Journal of Software Tools, 2000.
- The GPy authors, GPy: A Gaussian process framework in Python, 2012-2015.