Novel view generation using view morphing

[Seitz and Dyer, 1996]

Parallel views

• Suppose the camera has moved from the world origin to the position $(C_x,C_y,0)$ and the two projection matrices are
  $${\bf\Pi_0} = \left[ \begin{array}{cccc} f_0 & 0 & 0 & 0 \\ 0 & f_0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array} \right]$$
  and
  $${\bf\Pi_1} = \left[ \begin{array}{cccc} f_1 & 0 & 0 & -f_1C_x \\ 0 & f_1 & 0 & -f_1C_y \\ 0 & 0 & 1 & 0 \end{array} \right]$$

• Let ${\bf p_0}$ and ${\bf p_1}$ are image projections of a scene point ${\bf P}$ .
• Linear interpolation of ${\bf p_0}$ and ${\bf p_1}$ yields
  $$(1-s){\bf p_0} + s{\bf p_1} = (1-s)\frac{1}{Z}{\bf\Pi_0} {\bf P} + s\frac{1}{Z}{\bf\Pi_1} {\bf P} = \frac{1}{Z}{\bf\Pi_s} {\bf P}$$
  where
  $${\bf\Pi_s} = (1-s){\bf\Pi_0} + s{\bf\Pi_1}$$

• ${\bf\Pi_s}$ represents a camera with center $C_s =(sC_x,sC_y,0)$ and focal length $f_s = (1-s)f_0 + sf_1$ .

Linear morph is consistent with a physical pin-hole camera.

Non-parallel camera

• When two images share the optical center they are related by a planar homography.
• Thus there exist planar homographies between ${\cal I}_0 \leftrightarrow \hat{\cal I}_0$ , ${\cal I}_1 \leftrightarrow \hat{\cal I}_1$ and ${\cal I}_s \leftrightarrow \hat{\cal I}_s$ .
• View morphing can be achieved as a 3 step process.

The results also hold for uncalibrated cameras.

Monalisa