Affine Interpretation of the BS-Warps

Notation.

In this chapter, we denote  $ \mathcal {W}_B$ the BS-Warp. As a reminder of section 2.3.2, a BS-Warp is defined as:

$\displaystyle \mathcal {W}_B(\mathbf{q} ; \mathbf{x}) = \sum_{i=1}^m \sum_{j=1}^n \mathbf{p}_{ij} N_i(x) N_j(y),$ (6.2)

where the functions $ N_i : \mathbb{R}\rightarrow \mathbb{R}$ are the B-Spline basis functions and the $ \mathbf{p}_{ij} = (p_{ij}^x, p_{ij}^y)^\mathsf{T}$ are the control points (grouped in the parameter vector  $ \mathbf{x} \in \mathbb{R}^{2mn}$). The scalars $ m$ and $ n$ are the number of control points along the $ x$ and $ y$ directions respectively. Note that we consider as coincident the knot sequences used to define the B-Spline basis functions.

Affine interpretation.

If we consider that the observed surface is modelled by a threedimensional tensor-product B-Spline, the BS-Warp corresponds to the transformation between the two images under affine imaging conditions (see figure 6.1 for an illustration).
Figure 6.1: A BS-Warp can be seen as the result of a threedimensional B-Spline surface projected under affine conditions.
Image warp-affine

Let  $ \mathcal {R} : \mathbb{R}^2 \rightarrow \mathbb{R}^3$ be the 2D-3D map between the first image and the threedimensional surface:

$\displaystyle \mathbf{Q} = \mathcal {R}(\mathbf{q} ; \mathbf{x}) = \sum_{i=1}^m \sum_{j=1}^n \bar{\mathbf{p}}_{i,j} N_i(x) N_j(y),$ (6.3)

with $ \bar{\mathbf{p}}_{i,j} = (\bar{p}_{i,j}^x \: \bar{p}_{i,j}^y \: \bar{p}_{i,j}^z)^\mathsf{T}\in \mathbb{R}^3$ the 3D control points of the surface. Let  $ \mathcal {A}: \mathbb{R}^3 \rightarrow \mathbb{R}^2$ be the affine projection of the surface into the second image:

$\displaystyle \mathcal {A}(\mathbf{Q}) = \mathsf{A} \mathbf{Q},$ (6.4)

with  $ \mathsf{A}$ the matrix which models the affine projection, assuming that the 3D surface is expressed within the coordinate frame of the second camera:

$\displaystyle \mathsf{A} =
 \begin{pmatrix}
 a_x & 0 & 0  
 0 & a_y & 0  
 \end{pmatrix}.$ (6.5)

Given these notations, the warped point  $ \mathbf{q}'$ can be written  $ \mathcal {A} ( \mathcal {R} ( \mathbf{q} ) )$ which, after expansion, gives:

$\displaystyle \mathbf{q}' = \sum_{i=1}^m \sum_{j=1}^n \begin{pmatrix}\bar{p}_{i,j}^x  \bar{p}_{i,j}^y \end{pmatrix} N_i(x) N_j(y).$ (6.6)

Equation (6.6) matches the definition of equation (6.2) of a BS-Warp.

BS-Warps are not suited for perspective imaging conditions.

As we just demonstrated, BS-Warps are obtained under affine imaging conditions. However, this does not prove that they are not suited for perspective imaging conditions. In this paragraph, we experimentally illustrate that BS-Warps are indeed not suited for perspective imaging conditions. This comes from the fact that the division appearing in a perspective projection is not present in the BS-Warp model.

The bad behavior of the BS-Warp in the presence of perspective effects is illustrated in figure 6.2. In this experiment, we simulate a set of point correspondences by transforming a regular grid with an homography parametrized by a scalar $ a$ that controls the amount of perspective effect. The $ 3 \times 3$ matrix of this homography, $ \mathsf{H}_a$, is given by:

$\displaystyle \mathsf{H}_a \propto {\frac{1}{a}}
 \begin{pmatrix}
 (a+1)^2/4 & ...
...(a^2-1)/4  
 0 & a(a+1)/2 & 0  
 -(a^2-1)/4 & 0 & (a+1)^2/4
 \end{pmatrix},$ (6.7)

where $ \propto$ indicates equality up to scale. The larger $ \vert a-1\vert$, the more important the perspective effect. Figure 6.2 c clearly shows that the transformation can be correctly modelled by a BS-Warp only when $ a=1$, i.e. when the perspective effect is barely existant. Figure 6.2 d shows that the number of control points of a BS-Warp must be significantly large in order to correctly model a perspective effect.
Figure 6.2: Bad behavior of the BS-Warp in the presence of perspective effects. (a) Data points on a regular grid (first image). (b) Transformed points simulating a perspective effect with an homography (second image). (c) Influence of the perspective effect on a BS-Warp with 16 control points. (d) The perspective effect ( $ a=\frac{5}{2}$) can be modeled with a BS-Warp but it requires a large amount of control points.
Image homog1 Image homog2 Image badbs-a Image badbs-mn
(a) (b) (c) (d)


Contributions to Parametric Image Registration and 3D Surface Reconstruction (Ph.D. dissertation, November 2010) - Florent Brunet
Webpage generated on July 2011
PDF version (11 Mo)