How to show images in split window in OpenCV

A typical scenario in computer vision is to show multiple images in different parts of a single window, like a split window. However, OpenCV can only display a single cv::Mat in a single window using cv::imshow(). So, to achieve what we want, we create a single cv::Mat whose size is equal to the window. We copy our multiple images into this mat at the locations we want and just display the big mat.

(Note that this is far easier than the ROI regions you would need to use with IplImage.)

This sample illustrates showing two 640x480 images side-by-side in a single window:

// 640x480 images
cv::Mat mat_1;
cv::Mat mat_2;

// Create 1280x480 mat for window
cv::Mat win_mat(cv::Size(1280, 480), CV_8UC3);

// Copy small images into big mat
mat_1.copyTo(win_mat(cv::Rect(  0, 0, 640, 480)));
mat_2.copyTo(win_mat(cv::Rect(640, 0, 640, 480)));

// Display big mat
cv::imshow("Images", win_mat);

Tried with: OpenCV 2.4.8 and Ubuntu 14.04

Advertisements

Depth and type of matrix in OpenCV

cv::Mat is the most fundamental datatype used in OpenCV. It can be used to store 2D images with 1-4 channels of data.

When your code receives a cv::Mat from an external library or code, the most common question you have is what is the data type of the elements of this image? There seem to be two methods in the cv::Mat class that answer this: depth() and type().

Depth

Depth is the more fundamental of the two queries. It is the data type of each individual element in the image data. It can take one of these 8 values:

#define CV_8U   0
#define CV_8S   1
#define CV_16U  2
#define CV_16S  3
#define CV_32S  4
#define CV_32F  5
#define CV_64F  6
#define CV_USRTYPE1 7

For example, an image with 2 channels of unsigned short data will have depth of CV_16U.

Type

Type combines the data type of the elements along with the number of channels in the image. Since images in OpenCV can have 1-4 channels, it can take one of these 28 values:

#define CV_8UC1 CV_MAKETYPE(CV_8U,1)
#define CV_8UC2 CV_MAKETYPE(CV_8U,2)
#define CV_8UC3 CV_MAKETYPE(CV_8U,3)
#define CV_8UC4 CV_MAKETYPE(CV_8U,4)

#define CV_8SC1 CV_MAKETYPE(CV_8S,1)
#define CV_8SC2 CV_MAKETYPE(CV_8S,2)
#define CV_8SC3 CV_MAKETYPE(CV_8S,3)
#define CV_8SC4 CV_MAKETYPE(CV_8S,4)

#define CV_16UC1 CV_MAKETYPE(CV_16U,1)
#define CV_16UC2 CV_MAKETYPE(CV_16U,2)
#define CV_16UC3 CV_MAKETYPE(CV_16U,3)
#define CV_16UC4 CV_MAKETYPE(CV_16U,4)

#define CV_16SC1 CV_MAKETYPE(CV_16S,1)
#define CV_16SC2 CV_MAKETYPE(CV_16S,2)
#define CV_16SC3 CV_MAKETYPE(CV_16S,3)
#define CV_16SC4 CV_MAKETYPE(CV_16S,4)

#define CV_32SC1 CV_MAKETYPE(CV_32S,1)
#define CV_32SC2 CV_MAKETYPE(CV_32S,2)
#define CV_32SC3 CV_MAKETYPE(CV_32S,3)
#define CV_32SC4 CV_MAKETYPE(CV_32S,4)

#define CV_32FC1 CV_MAKETYPE(CV_32F,1)
#define CV_32FC2 CV_MAKETYPE(CV_32F,2)
#define CV_32FC3 CV_MAKETYPE(CV_32F,3)
#define CV_32FC4 CV_MAKETYPE(CV_32F,4)

#define CV_64FC1 CV_MAKETYPE(CV_64F,1)
#define CV_64FC2 CV_MAKETYPE(CV_64F,2)
#define CV_64FC3 CV_MAKETYPE(CV_64F,3)
#define CV_64FC4 CV_MAKETYPE(CV_64F,4)

For example, an image with 2 channels of unsigned short data will have type of CV_16UC2.

Here are two simple functions that are handy to get the depth and type of image as a string:

Reference: modules/core/include/opencv2/core/hal/interface.h in OpenCV source code

How to apply mask in OpenCV

In OpenCV, a mask image is of type uint8_t. Pixels of value 0xFF are true and pixels of value 0 are false.

A mask can be applied on an image of the same dimensions, but of any type. By applying a mask M on an image I, the pixels of I whose corresponding pixel in M are true are copied into a new image. The rest of the pixels in the new image are set to 0.

Applying a mask in OpenCV is easy:

cv::Mat in_mat;   // Already created
cv::Mat mask_mat; // Already created
cv::Mat out_mat;  // New and empty

in_mat.copyTo(out_mat, mask_mat);

Note that the input and output image should not be the same! OpenCV does not throw an error, but the behavior is undefined.

Tried with: OpenCV 2.4.8 and Ubuntu 14.04

Eigenvectors and eigenvalues in OpenCV

There are many libraries for C++ that can be used to compute Eigenvectors and Eigenvalues. However, if you are already using OpenCV in your project, then it can be used to get these values easily by using the PCA class. This class is declared in opencv2/core/core.hpp header file and defined in the modules/core/src/matmul.cpp source file.

Here is an example:

Tried with: OpenCV 2.4.9 and Ubuntu 14.04

Fonts in OpenCV

Fonts available in OpenCV
Fonts available in OpenCV

OpenCV can be used to render text on an image buffer using the putText function. Several simple fonts are available in OpenCV which can be used to write text.

The available fonts can be seen defined in modules/core/include/opencv2/core/core.hpp header file:

enum
{
    FONT_HERSHEY_SIMPLEX = 0,
    FONT_HERSHEY_PLAIN = 1,
    FONT_HERSHEY_DUPLEX = 2,
    FONT_HERSHEY_COMPLEX = 3,
    FONT_HERSHEY_TRIPLEX = 4,
    FONT_HERSHEY_COMPLEX_SMALL = 5,
    FONT_HERSHEY_SCRIPT_SIMPLEX = 6,
    FONT_HERSHEY_SCRIPT_COMPLEX = 7,
    FONT_ITALIC = 16
};

FONT_ITALIC is not a font, but can be combined with the other fonts to get italic text.

Tried with: OpenCV 2.4.9 and Ubuntu 14.04

How to specify OpenCV color type in Python

Problem

OpenCV has bindings for Python. Specifying the color type in calls to imread or imdecode gives this error:

$ cat foo.py
import cv
import cv2
mat = cv2.imread("foo.png", cv.CV_LOAD_IMAGE_ANYCOLOR)

$ python foo.py
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'

Solution

It looks like the CV_LOAD_IMAGE enums which specify the color type of an image have not been exported to Python. Thankfully, the integer value of these enums can be passed directly to the call.

This enum is defined in highgui/highgui_c.h as:

enum
{
/* 8bit, color or not */
    CV_LOAD_IMAGE_UNCHANGED  =-1,
/* 8bit, gray */
    CV_LOAD_IMAGE_GRAYSCALE  =0,
/* ?, color */
    CV_LOAD_IMAGE_COLOR      =1,
/* any depth, ? */
    CV_LOAD_IMAGE_ANYDEPTH   =2,
/* ?, any color */
    CV_LOAD_IMAGE_ANYCOLOR   =4
};

Make the call using the integer value of the enum directly:

# To call with CV_LOAD_IMAGE_ANYCOLOR
mat = cv2.imread("foo.png", 4)

Tried with: OpenCV 2.4.9, Python 2.7.6 and Ubuntu 14.04

How to use OpenCV waitKey in Python

The cv::waitKey(n) function in OpenCV is used to introduce a delay of n milliseconds while rendering images to windows. When used as cv::waitKey(0) it returns the key pressed by the user on the active window. This is typically used for keyboard input from user in OpenCV programs.

char c = cv::waitKey(0);
if ('q' == c)
    QuitProgram();

The above function is available in the Python API of OpenCV. However, it seems to return an integer value that cannot be compared to a char like above. In fact, it is an ASCII value only in the last 8 bits. To use it to compare with a char first mask everything but the first 8 bits and then convert it using the chr builtin method of Python:

c = cv2.waitKey(0)
if 'q' == chr(c & 255):
    QuitProgram()

Tried with: OpenCV 2.4.9, Python 2.7.3 and Ubuntu 14.04

How to use OpenCV constants in Python

OpenCV has a Python API. The Python function calls are well documented and listed along with the C and C++ calls. However, it is not obvious how to use the various OpenCV constants and enums in Python. These are typically named as CV_SOME_THING. For example: CV_WINDOW_AUTOSIZE.

  • A few of these named constants seem to be available for use in Python by importing the cv package. For example:
import cv
win = cv.NamedWindow("haha", cv.CV_WINDOW_AUTOSIZE)
  • However, I find that many of these named constants are not available in Python. You will need to look up their integer value from OpenCV header files and pass those raw integer values directly to the calls.

Here is a list of integer values for OpenCV constants that I frequently need to use:

CV_LOAD_IMAGE_UNCHANGED = -1
CV_LOAD_IMAGE_GRAYSCALE =  0
CV_LOAD_IMAGE_COLOR     =  1
CV_LOAD_IMAGE_ANYDEPTH  =  2
CV_LOAD_IMAGE_ANYCOLOR  =  4

Tried with: OpenCV 2.4.9, Python 2.7.3 and Ubuntu 14.04

How to convert between image formats in OpenCV

  • The cv::cvtColor function call can be used to convert between different image formats in OpenCV.

  • The header file to include is opencv2/opencv.hpp.

  • The library file to link with on Windows is opencv_imgproc249.lib for Release mode and opencv_imgproc249d.lib for Debug mode. This assumes that I am using OpenCV 2.4.9.

  • As an example of usage, to convert from RGB to BGR format:

cv::Mat in_mat; // Holds RGB image
cv::Mat out_mat; // Output in BGR format
cv::cvtColor(in_mat, out_mat, CV_RGB2BGR);
  • CV_RGB2GRAY convert 3-channel RGB image to 1-channel grayscale image.

Tried with: OpenCV 2.4.9 and Windows 7 Professional 64-bit