NURBS-Warps

This section introduces a new warp, the NURBS-Warp, which is built upon tensor-product Non-Uniform Cubic B-Splines. We show that the NURBS-Warp naturally appears when replacing the affine projection by a perspective one in the image formation model of the previous section. As our experimental results will show, the NURBS-Warp performs better in the presence of perspective effects. All the necessary details about the NURBS model have been presented in section 2.3.3.

Perspective interpretation.

Following the same reasoning than for the BS-Warp in the previous section, we show that the NURBS-Warp corresponds to perspective imaging conditions. This is illustrated in figure 6.3.

Figure 6.3: A NURBS-Warp can be seen as the result of a threedimensional B-Spline surface projected under perspective conditions.

Let  $\mathcal {P}$ be the perspective projection:

$$\mathcal {P}(\mathbf{Q}) = \psi(\mathsf{K} \mathbf{Q}),$$ (6.8)

with  $\mathsf{K}$ the matrix of intrinsic parameters for the second camera. $\psi$ is the homogeneous to affine coordinates function, i.e. $\psi(\check{\mathbf{q}}) = \mathbf{q}$ where  $\check{\mathbf{q}}$ are the homogeneous coordinates of  $\mathbf{q}$ ( $\check{\mathbf{q}}^\mathsf{T}= (\mathbf{q}^\mathsf{T}\; 1)$). We assume that the image coordinates are chosen such that the origin coincides with the principal point:

$$\mathsf{K} = \begin{pmatrix} f_x & 0 & 0 0 & f_y & 0 0 & 0 & 1 \end{pmatrix}.$$ (6.9)

Replacing  $\mathbf{Q}$ by its expression of equation (6.3) in equation (6.8) leads to:

$$\mathcal {P}(\mathbf{Q}) = \mathcal {P}(\mathcal {R}(\mathbf{q}))... ...f_y \sum_{i=1}^m \sum_{j=1}^n \bar{p}_{i,j}^y N_i(x) N_j(y) \end{pmatrix}.$$ (6.10)

Defining $w_{i,j}=\bar{p}_{i,j}^z$, $p_{i,j}^x = f_x\frac{\bar{p}_{i,j}^x}{w_{i,j}}$ and $p_{i,j}^y = f_y\frac{\bar{p}_{i,j}^y}{w_{i,j}}$, equation (6.10) is the very definition of a tensor-product NURBS with control points  $\mathbf{p}_{i,j}^\mathsf{T}= (p_{i,j}^x, p_{i,j}^y)$ and weights $w_{i,j}$ (see section 2.3.3). We denote  $\mathcal {W}_N$ this new warp and call it a NURBS-Warp:

$$\mathcal {W}_N(\mathbf{q} ; \mathbf{x}) = \frac{\sum_{i=1}^m \sum... ..._{i,j} w_{i,j} N_i(x) N_j(y)}{\sum_{i=1}^m \sum_{j=1}^n w_{i,j} N_i(x) N_j(y)}.$$ (6.11)

Here, the warp parameters, i.e. the control points and the weights, are grouped into a vector  $\mathbf{x} \in \mathbb{R}^{3mn}$.

Using the NURBS-Warp in the setup used for the experiment of figure 6.2 leads to a transfer error consistently smaller than $10^{-5}$ pixels.

Homogeneous NURBS-Warp.

The NURBS-Warp defined by equation (6.11) can be expressed with homogeneous coordinates. We note  $\check{\mathcal {W}}_N$ the NURBS-Warp in homogeneous coordinates:

$$\check{\mathcal {W}_N}(\mathbf{q} ; \mathbf{x}) \propto \begin... ...j}^x w_{i,j} p_{i,j}^y w_{i,j} w_{i,j} \end{pmatrix} N_i(x) N_j(y).$$ (6.12)

We observe that in the homogeneous version of equation (6.12), our NURBS-Warp does a linear combination of control points in homogeneous coordinates, as opposed to the classical BS-Warp of equation (6.2) that does a linear combination of control points in affine coordinates. This is what makes our NURBS-Warp able to model perspective projection, thanks to the division `hidden' in the homogeneous coordinates.