Method and system for generating panoramic images from video sequences

• US 20060062296A1
• Filed: 11/18/2003
• Published: 03/23/2006
• Est. Priority Date: 11/26/2002
• Status: Abandoned Application

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Xu, Li-Qun, Morphett, Jason, Nightingale, Charles, Li, Yongmin, Morrison, David G
• Application Number:
  US10/535,634
• Publication Number:
  US 20060062296A1
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The invention relates to generating panoramic images of foreground and background panorama. Within mosaiced panoramic images there will often be more than one pixel value available for each pixel position. In the invention pixel values for a foreground panoramic image are selected by taking the most extraordinary pixel as foreground, and pixel values for a background image are selected by taking a median pixel value. We have found that such pixel selection gives improved results with respect to the prior art.

Claims

1. A method of generating panoramic images from a motion-compensated inter-frame encoded video sequence, the method comprising:

positioning the image of each frame of the sequence on a panoramic image reference plane, such that the respective images of each frame are in registration with each other, and one or more pixel values are available for each pixel position in the reference plane; and

for each pixel position in the reference plane, selecting one of the available pixel values for use as the pixel value in the panoramic image;

the method being characterised in that the selecting step further comprises selecting a substantially median pixel value from the available pixel values for use in a background panoramic image and/or selecting a substantially most different pixel value from the available pixel values for use in a foreground panoramic image.

2. A method according to claim 1, wherein the positioning step further comprises:

determining global motion estimations for each frame in the sequence;

selecting a particular frame of the sequence as a reference frame, the plane of the reference frame being the panoramic image reference plane;

for each frame other than the reference frame, accumulating the global motion estimations from each frame back to the reference frame; and

warping each frame other than the reference frame onto the reference plane using the accumulated global motion estimations to give one or more pixel values for each pixel position in the reference plane.

3. A method according to any of claims 1, wherein the selecting step further comprises:

calculating the mean pixel value of the available pixel values;

calculating the L1 distance between each available pixel value and the calculated mean pixel value; and

selecting the pixel value with the median L1 distance for use in a background panoramic image; and

/or

selecting the pixel value with the maximum L1 distance for use in a foreground panoramic image.

4. A computer program or suite of programs arranged such that when executed on a computer system the program or suite of programs causes the computer system to perform the method of claim 1.

5. A computer readable storage medium storing a computer program or suite of programs according to claim 4.

6. A system for generating panoramic images from a motion-compensated inter-frame encoded video sequence, comprising:

image registration means for positioning the image of each frame of the sequence on a panoramic image reference plane, such that the respective images of each frame are in registration with each other, and one or more pixel values are available for each pixel position in the reference plane; and

pixel selection means arranged in use, for each pixel position in the reference plane, to select one of the available pixel values for use as the pixel value in the panoramic image;

the system being characterised in that the pixel selection means further comprises background pixel selection means for selecting a substantially median pixel value from the available pixel values for use in a background panoramic image and/or foreground pixel selection means for selecting a substantially most different pixel value from the available pixel values for use in a foreground panoramic image.

7. A system according to claim 6, wherein the image registration means further comprises:

motion estimator means for determining global motion estimations for each frame in the sequence;

frame selector means for selecting a particular frame of the sequence as a reference frame, the plane of the reference frame being the panoramic image reference plane;

motion estimation accumulator means for, for each frame other than the reference frame, accumulating the global motion estimations from each frame back to the reference frame; and

frame warping means for warping each frame other than the reference frame onto the reference plane using the accumulated global motion estimations to give one or more pixel values for each pixel position in the reference plane.

8. A system according to claim 6, wherein the background pixel selection means further comprises:

calculator means arranged in use to;

calculate the mean pixel value of the available pixel values; and

calculate the L1 distance between each available pixel value and the calculated mean pixel value; and

a median pixel selector arranged to select the pixel value with the median L1 distance for use in a background panoramic image

9. A system according to claim 6, wherein the foreground pixel selection means further comprises:

calculator means arranged in use to;

calculate the mean pixel value of the available pixel values; and

calculate the L1 distance between each available pixel value and the calculated mean pixel value; and

a maximum pixel selector arranged to select the pixel value with the maximum L1 distance for use in a foreground panoramic image.