Original image sequence

If you take a look at the original image sequence (presented here in MPEG),

you can see that it wraps back on itself, that is, the camera changes direction, so that the last image in the sequence overlaps substantially with the first.

If we mosaic the images in the usual manner, by estimating the coordinate transformation that takes each frame back to the reference frame (say the first frame, which we might call frame0), we see that the shape of the frame changes .........

There are two basic methods of determining the coordinate transformation between the images in an image sequence:

Once these coordinate transformations are estimated, we may apply them to each image, so that we obtain a stabilized image sequence.

When we register the first two images, in accordance with the estimated parameters, some of the image data from the comparison frame (the second frame) extends beyond the boundaries of the reference frame (the first frame). "Cementing" these two images together provides us with an irregularly shaped image:

, to which we can cement a third image,

enlarging the "canvas" a little bit each time a new image arrives. Continuing in this manner, cementing each new frame into a growing "canvas" (the bounding box of the mosaic grows with each new arriving image), we arrive at a completed mosaic:

However, we observe that this image shows quite a bit of misregistration, which arose from cumulative error, as well as a "quilting" effect where the boundaries of image frames are evident. We can attempt to hide the misregistration by applying an increasing weighting to newer frames as they arrive. For example, we may apply an exponential weighting, as in the following image:

We may also use a fuzzy crossover near the edges of the images to remove the "quilting" effect:

These two steps together would produce a visually acceptable mosaic, but we can do better....


support matrix, cumulative error...

Submosaics arranged as indicated by cumulative parameter estimation:

Final mosaic:

Comparison with previous affine methods (Salient Stills)

In order to make a "fair" comparison, the existing unmodified Salient Stills software was used, rather than being re-implemented for this comparison.


The result is not particularly good, as the motion estimator failed to track the motion. With some "tweaking" of the parameters, it was possible to get a better-looking image, but still not as good as the proposed projective method.


-noshear -nozoom

-noshear -nozoom -i 2 3 4 5 6

return to Video Orbits of the Projective Group: A New Perspective on Image Mosaicing.