Abstract

In this article we present a framework for real-time mosaicing from video sequences recorded from an uncalibrated PTZ camera based on multiframe registration. To this end, a new frame alignment algorithm, the Direct Local Indirect Global (DLIG), is presented.

The key idea of the DLIG alignment is to divide the frame alignment problem into the problem of registering a set of spatially related image patches. The registration is iteratively computed by sequentially imposing a good local match and global spatial coherence. The patch registration is performed using a tracking algorithm, so a very efficient local matching can be achieved. We use the patch-based registration to obtain multiframe registration, using the mosaic coordinates to relate the current frame to patches from different frames that partially share the current Field of View. Multiframe registration prevents the error accumulation problem, one of the most important problems in mosaicing. We also show how to embed a Kernel Tracking algorithm in order to obtain a precise and extremely efficient mosaicing algorithm. Finally, we perform a quantitative evaluation of our algorithm, including a comparison with other alignment approaches, and studying its performance against interlaced videos and illumination changes.

Index Terms - Video mosaicing, real-time, multiframe alignment, interlaced, kernel tracking