Given a set of input images such as:
consider the problem of genrating a novel view.
- Finely sample the back-projected ray from a site in the target super resolved
view between a range
. For each
in this range, project the 3D point
to each of the low resolution input images and compute
as a set of possible colours corresponding to depth
- For each
do a nearest neighbour clustering of
space, with a suitable distance threshold on colour differences, and remove all but the top few dominant clusters. If there are no dominant clusters then we
from the list of candidate depths at
- For all candidate
is the number of dominant clusters, and
is the set of the image indices that belong to the
For each depth
- Define a correlation function
is the set of
patches around the low resolution pixel
is the sum of pair-wise correlations between patches of
- Form a set of candidate depths
by choosing the local minimas of
- More than one dominant clusters at any
- Resolve occlusion using the ordinal visibility constraint:
any two points on two back projected rays then
- Novel view result:
(a) One of the input images (out of 40). (b) Photo-consistent depth map, note that the depth map
is not accurate at occluding pixels and at homogeneous parts.
(c) Occlusion-map: the pixels for which multiple clusters are formed, and are marked white.
(d) The generated novel view.
Note that though the depth map
generated is not accurate, it provides correct photo-consistent colour in the novel view.
Generated novel view and the difference image of novel view with the ground truth.