Method and apparatus for automatic electronic replacement of billboards in a video image

• US 6,208,386 B1
• Filed: 05/13/1997
• Issued: 03/27/2001
• Est. Priority Date: 09/08/1995
• Status: Expired due to Term

• Current Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Original Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Inventors: Tamir, Michael, Wilf, Itzhak, Sharir, Avi
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US08/776,038
• Time in Patent Office: 1,414 Days
• Field of Search: H04/N.5262, H04/N.5225, 348/744, 348/578, 348/157, 348/143, 348/135, 348/137, 348/140, 348/580, 348/586, 348/587, 348/592, 358/906, 386/107, 386/117
• US Class Current: 348/578
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

Apparatus for automatic electronic replacement of a billboard in a video image including an automatic camera orientation measurement apparatus including motion measurement means operative to measure the Field of View (FOV) of the TV camera relative to a known reference position.

Claims

1. Apparatus for automatic replacement of a billboard in a video image comprising:

an automatic camera orientation measurement apparatus including motion measurement means operative to measure the Field of View (FOV) of the TV camera relative to a known reference position; and

an image processing means for processing video signals generated by the TV camera, wherein said processing means includes calibration means for periodically automatically calibrating the motion measurement means, the calibration means including a fine adjustment image processing means for perspectively transforming the current video image to provide an estimated transformed model, means for storing a replica of the billboard image, means for comparing the perspectively transformed video image with the stored replica to provide a residual video field, means for solving residual distortion between the transformed model and the residual video field and means for updating the estimated transformation by means of the residual distortion to provide a calibration correction factor.

2. Apparatus as claimed in claim 1, wherein the motion measurement means includes means for measuring the pan tilt, zoom and focus of the camera relative to known reference positions.

3. Apparatus as claimed in claim 1, further comprising a billboard in which said billboard comprises a chroma-key panel.

4. Apparatus as claimed in claim 3, wherein the panel comprises a patterned chroma-key panel.

5. Apparatus as claimed in claim 4, wherein the pattern comprises two different colors or two different shades of the same color.

6. Apparatus as claimed in claim 5, wherein the pattern is designed to be optimized for use according to the size and shape of the billboard and to the anticipated video conditions.

7. Apparatus as claimed in claim 1, wherein said means for solving residual distortion includes analysis means based in the spatial derivatives of the current image of the transformed model as well as means for analyzing pixel by pixel the image differences of the transformed model and the current image.

8. Apparatus as claimed in claim 1, wherein the fine adjustment means includes means for carrying out the fine adjustment in an interactive manner.

9. Apparatus as claimed in claim 1, wherein the pixel by pixel analysis means includes means for discarding pixels using color variation.

10. Apparatus as claimed in claim 1, further comprising means for recording calibration data obtained during a set up period when no occlusion is present.

11. Apparatus as claimed in claim 1 further comprising a dynamic billboard memory for recording any changes in measured position of the billboard as the camera pans or tilts.

12. A method of automatic replacement of a billboard in a video image said method comprising the steps of:

recording the Field of View (FOV) parameters of a TV camera in a first reference position;

storing the image viewed by the TV camera in said first reference position;

measuring the movement of the camera relative to the first reference position and recording the measurements and periodically automatically calibrating the motion measurements by comparison of the present video image produced by said TV camera with the stored video image in said reference position which comparison includes a fine adjustment image processing step including perspectively transforming the current video image to provide an estimated transformed model, storing a replica of the billboard image, comparing the perspectively transformed video image with the stored replica to provide a residual video field, solving residual distortion between the transformed model and the residual video field and updating the estimated transformation by means of the residual distortion to provide a calibration correction factor; and

calibrating a difference signal for recalibration of the motion measurements and said first reference position and replacing a real billboard in said camera'"'"'s field of view with a virtual replacement billboard in accordance with said recalibrated first reference position.

13. The method as claimed in claim 12, wherein said step of measuring the movement of the camera relative to the first reference position comprises the steps of measuring the pan, tilt, zoom or focus of the camera by means of sensors attached to the camera and the camera lens.

14. The method as claimed in claim 12, wherein the billboard comprises a chroma-key panel and in which said step of replacement of the real billboard by the virtual billboard includes the step of occluding said replacement billboard by any object occluding said real billboard.

15. A method as claimed in claim 12, wherein said step of solving the residual distortion includes analysis based in the spatial derivatives of the current image of the transformed model and analyzing pixel by pixel the image differences of the transformed model and the current image.

16. The method as claimed in claim 12, wherein the fine adjustment step comprises carrying out the fine adjustment in an interactive manner.

17. method as claimed in claim 15, wherein the pixel by pixel analysis step includes discarding pixels by using color variation.

18. method as claimed in claim 12, wherein step of recording calibration data is carried out during a set up period when no occlusion is present.

19. A method as claimed in claim 12 in which said step of calibrating a difference signal is carried out only when the present video image is above a predetermined quality factor.

20. A method as claimed in claim 19 in which said virtual replacement billboard is positioned at the last recalibrated position in the event that a recalibration is not possible due to the present video image being below the predetermined quality factor.