14. Image Features¶

Reading Assignment

Please read section from 13.1.3 of [RVC] and chapter 18 of [PIVP].

Video Resources

Peter Corke’s Image Features Series

The iblobs function from the MVTB finds image blobs like the ilabel command, but it also calculates a number of features derived from the size and shape of each blob. It returns an array of RegionFeature objects with properties for the blob features.

Please refer to the [RVC] and Dr. Corke’s videos for full details about how the features are calculated. I’ll just highlight a few points here and will also review it in class.

im = iread('sharks.png');
>> blobs = iblobs(im, 'touch', 0, 'boundary')
blobs =
(1) area=14899, cent=(298.0,181.0), theta=1.48, b/a=0.702, class=1, label=1, touch=0, parent=3, perim=1236.4, circ=0.136
(2) area=7728, cent=(503.5,184.7), theta=-0.54, b/a=0.558, class=1, label=2, touch=0, parent=3, perim=515.8, circ=0.406
(3) area=18814, cent=(246.8,426.9), theta=-0.02, b/a=0.559, class=1, label=4, touch=0, parent=3, perim=812.2, circ=0.399
(4) area=7746, cent=(84.2,160.7), theta=0.50, b/a=0.558, class=1, label=5, touch=0, parent=3, perim=514.4, circ=0.409

14.1. Moments¶

The moments of a blob are simple calculations that can be used to determine features related to the size, shape and orientation of a blob. The moment equation is:

$m_{pq}=\sum_{(u,v)\in \boldsymbol{\mathrm{I}}}{u^pv^q\boldsymbol {\mathrm{I}}[u,v]}$

A particularly interesting moment is when $p$ and $q$ equals 0. We call this the zeroth moment of the blob, which corresponds to total count of the number of pixels in the blob.

$m_{00}=\sum_{(u,v)\in \boldsymbol{\mathrm{I}}}{\boldsymbol {\mathrm{I}}[u,v]}$

The first moments are the moments 1,0 and the moment 0,1. If you have studied static forces on a rigid body in statics or physics class, then you have used first moments. The first moment in statics is a sum of the products of forces and their distances from a fixed point. The same concept applies here except instead of using a magnitude of a force we count the number of pixels for each $u$ or each $v$ .

$m_{10}=\sum_{(u,v)\in \boldsymbol{\mathrm{I}}}{u\boldsymbol {\mathrm{I}}[u,v]}$

$m_{01}=\sum_{(u,v)\in \boldsymbol{\mathrm{I}}}{v\boldsymbol {\mathrm{I}}[u,v]}$

In the statics of a rigid body, the center of the mass is the first moment divided by the total mass. Likewise, we can find the centroid of a blob.

$u_c=\frac{m_{10}}{m_{00}},\ v_c=\frac{m_{01}}{m_{00}}$

The MVTB function mpq calculates the moments.

>> im = iread('shark1.png');
>> idisp(im)
>> m00 = mpq(im, 0, 0)
m00 =
    7827
>> m10 = mpq(im, 1, 0)
m10 =
    1348568
>> m01 = mpq(im, 0, 1)
m01 =
    1222178
>> uc = m10 / m00
uc =
    172.2969
>> vc = m01 / m00
vc =
    156.1490
>> plot_point([uc vc]', 'o')

>> blobs = iblobs(im, 'touch', 0, 'boundary')
blobs =
(1) area=7827, cent=(172.3,156.1), theta=-0.21, b/a=0.585, class=1, label=2, touch=0, parent=1, perim=557.6, circ=0.352
>> blobs(1).uc
ans =
    172.2969
>> blobs(1).vc
ans =
    156.1490

14.2. Equivalent Ellipse¶

To find features of the blob that describe the general shape and orientation, we find an equivalent ellipse to the blob. That is, we wish to find an ellipse that generally fits the blob in terms of aspect ratio (ratio of length to height) and orientation. We do this by matching the inertia matrix of the blob to the equivalent ellipse. The inertia matrix comes from the moments.

The following are blob properties derived from the equivalent ellipse.

>> blobs(1).area
ans =
        7827
>> blobs(1).theta
ans =
    -0.2123
>> blobs(1).aspect
ans =
    0.5848
>> blobs(1).a
ans =
    70.4348
>> blobs(1).b
ans =
    41.1874

The variable a is the long length of the ellipse and b is the short length. So the ratio $b/a$ is called the aspect ratio, ( $aspect \le 1$ ).

Warning

Advanced math ahead

See the book for equations relating to the inertia matrix and how to calculate the equivalent ellipse dimensions and rotations. The length of the ellipse’s major and minor axis come from the eigenvalues of the inertia matrix. The orientation comes the eigenvectors of the inertia matrix.

The eigenvectors and eigenvalues of a matrix is a linear algebra topic covered in the Data Analysis and Tools class. You might want to look at the study guide for that class. Given a matrix $\bm{A}$ , the eigenvectors are a special vector that when multiplied by $\bm{A}$ , yield a scalar ( $\lambda$ ) multiple of the vector. Thus the following relationship holds for the eigenvectors and eigenvalues of a matrix.

$\bm{A\,x} = \lambda\,\bm{x}$

This special relationship has many applications – the example here, being one of many.

To learn more about this topic, search for information about the principal axis theorem and inertia matrix of an ellipse.

In the following example, we illustrate finding the equivalent ellipse for a 100-by-200 pixel blob rotated 45 degrees.

>> im = zeros(501);
>> im(200:300, 150:350) = ones(101, 201);
>> im = imrotate(im, 45, 'bilinear','crop');
>> idisp(im)
>> blob = im > 0;
>> m00 = mpq(blob, 0, 0);   % Zero moment

% central moments with respect to the centroid
>> u20 = upq(blob,2,0); u02 = upq(blob,0,2); u11 = upq(blob, 1,1);
>> J = [ u20 u11; u11 u02]; % inertia matrix

>> plot_ellipse(4*J/m00, [251, 251], 'b')

>> [x, lambda] = eig(J); % eigenvectors and eigenvalues
>> lambda
lambda =
1.0e+07 *
    1.7791         0
        0    7.0180

% Major, Minor axis and aspect ratio
>> a = 2 * sqrt(lambda(2,2) / m00)
a =
  116.7583
>> b = 2 * sqrt(lambda(1,1) / m00)
b =
  58.7863
>> b/a
ans =
    0.5025

>> x
x =
-0.7071   -0.7071
-0.7071    0.7071
>> v = x(:,end);
>> atand(v(2)/v(1))
ans =
-45.0028

You may wonder why the orientation is listed as a negative angle when we rotated it counter-clockwise. The answer has to do with the v axis. When MATLAB displays an image, values of the v axis increase coming down from the top. If we plot the blob as a regular plot, we see that the blob has a negative angle orientation.

>> [v, u] = find(blob);
>> plot(u,v, '.')

14.3. Circularity¶

If the option 'boundary' is given to the iblobs function, it also calculates the circumference (perimeter) of the blob and returns the following additional properties.

>> blobs(1).perimeter
ans =
    557.5706
>> blobs(1).circularity
ans =
    0.3520

Circularity is very useful for determining the general shape of the blob. It is given by:

$\frac{4\,\pi\,area}{(perimeter)^2}$

Calculate the circularity for a circle, square, rectangle, and triangle to see how circularity can indicate the shape of a blob.

14.4. Filtering Blobs¶

The iblobs function can filter the results to return only blob RegionFeature objects that satisfy specified filter rules. The filtering options are:

'area',[A1,A2]: accept only blobs with area in the interval A1 to A2

'aspect',[S1,S2]: accept only blobs with aspect ratio in the interval S1 to S2

'touch',T: accept only blobs that touch (1) or do not touch (0) the edge (default accept all)

'class',C: accept only blobs of pixel value C (default all)

14. Image Features¶

14.1. Moments¶

14.2. Equivalent Ellipse¶

14.3. Circularity¶

14.4. Filtering Blobs¶

Table of Contents

Previous topic

Next topic

This Page