2. Key Deformations and Retargeting - Cartoons, .. will deform like cartoons !!

Now we need to capture the local deformations the cartoon undergoes in order to be able to transfer the same deformation to the target.

1. Preprocessing -
   • The cartoon whose motion is to be captured is to be seperated from the background.
   • This was done using a color based region growing segmentation.
   • After this the silhouette of the segmented figure is made and we work with this silhouette.
   
   
   
2. Correspondence and Pairwise Morphing -
   • Now the problem is that the space formed by the animation frames is generally sparse and not suitable for interpolation so we need to densely populate this shape space. So we generate frames between each keyframe pair.
   • Since a tracker was not used to track points across frames, the point correpondences between each pair of keyframes had to be clicked manually as in affine capture.
   • Then we triangulate the clicked points using delaunay triangulation. Now we generate the inbetween frames using standard (linear) morphing technique by specifying suitable weights.

     Frame n	Frame n+1

   • Then we calculate the weight of the generated frame as explained below. If the weight comes out to be the same as the weight used to generate the frame then we stop else we iterate using different threshold values (to reject outliers formed due to ghosting artefacts in linear morphs) till we converge.
   
   
   
3. Calculation of Weights -
   • We calculate the weights for the frames inbetween two keyframes. The weight will be used as an interpolant between the keyframes to obtain the frame in question.
   • If Ii be the first keyframe, Ij be the second keyframe, and It be the keyframe whose weight we are estimating then the weight can be obatined by minimizing the error quantity err given by -
     

     err = [ Ij - ( w.Is + ( 1.0 - w ).It ) ]²

   • Note we are assuming a linear pairwise interpolation here, as opposed to the nonlinear space interpolation in the original paper. We will see the limitations of this assumption.
   
   
   
4. Retargeting -
   • Retargeting involes triangulating the target keyframes and generating the inbetween frames using the estimated weights. We then tranfer the affine component of the capture as discussed in the previous section.
   
   
   
5. Problems -
   • Firstly linear interpolation is used beacuse it is simple. It works well for cases where the deformations are small. For larger deformations it fails.
   • Convergence of the iterative process is hard in cases of large deformations.
   • The calculated weight is minimizing a intensity based error function, while in morphing the weight is used to linearly warp a triangulation. Ideally the error should also minimize a triangulation warping error in such a case.
   • The process requires a lot of intuitive manual work.

Page last updated on 04 October, 2005.	AT cse.iitd.ac.in	© Parag Chaudhuri , 2009