Method and apparatus for automatic electronic replacement of billboards in a video image

• US 6,384,871 B1
• Filed: 11/01/2000
• Issued: 05/07/2002
• Est. Priority Date: 09/08/1995
• Status: Expired due to Fees

• Current Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Original Assignee: Orad Hi-Tec Systems Ltd. (Avid Technology Incorporated)
• Inventors: Wilf, Itzhak, Tamir, Michael, Sharir, Avi
• Primary Examiner(s): Eisenzopf, Reinhard J.
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US09/703,442
• Time in Patent Office: 552 Days
• Field of Search: H04/N.5262, H04/N.5225, 348/578, 348/744, 348/157, 348/143, 348/135, 348/137, 348/140, 348/580, 348/586, 348/587, 348/592, 348/39, 386/107, 386/117, 358/906
• US Class Current: 348/578
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02
• CPC Class Codes: H04N 9/75 Chroma key

Abstract

Apparatus for automatic electronic replacement of a billboard in a video image including an automatic camera orientation measurement apparatus including motion measurement means operative to measure the Field of View (FOV) of the TV camera relative to a known reference position.

Claims

1. Apparatus for automatic replacement of a billboard in a video image, comprising:

(i) a camera position store that receives input from sensors of a camera that are indicative of a position of the camera and that stores the input;

(ii) a billboard position store that stores a position of the billboard;

(iii) a processor that determines from the stored input in the camera position store and the stored position of the billboard in the billboard position store and independently of a video signal generated by the camera whether the billboard is within the field of view of the camera; and

(iv) a replacement billboard store that stores a replacement billboard for replacing the billboard when the processor determines that the billboard is within the field of view of the camera.

2. Apparatus for automatic replacement of a billboard in a video image as claimed in claim 1, further comprising a calibration means to periodically automatically calibrate the sensors.

3. Apparatus as claimed in claim 1, wherein the sensors comprise means for measuring the pan tilt, zoom and focus of the camera relative to known reference positions.

4. Apparatus as claimed in claim 2, wherein the sensors comprise means for measuring the pan tilt, zoom and focus of the camera relative to known reference positions.

5. Apparatus as claimed in claim 2, wherein the billboard comprises a chroma-key panel.

6. Apparatus as claimed in claim 5, wherein the panel comprises a patterned chroma-key panel.

7. Apparatus as claimed in claim 6, wherein the pattern comprises two different colors or two different shades of the same color.

8. Apparatus as claimed in claim 7, wherein the pattern is designed to be optimized for use according to the size and shape of the billboard and to the anticipated video conditions.

9. Apparatus as claimed in claim 2, wherein the calibration means includes fine adjustment image processing means including means for perspectively transforming the current video image to provide an estimated transformed model, means for storing a replica of the billboard image the perspectively transformed video image with the stored replica to provide a residual video field, means for solving residual distortion between the transformed model and the residual video field and means for updating the estimated transformation by means of the residual distortion to provide a calibration correction factor.

10. Apparatus as claimed in claim 9, wherein said means for solving residual distortion includes analysis means based in the spatial derivatives of the current image of the transformed model as well as means for analyzing pixel by pixel the image differences of the transformed model and the current image.

11. Apparatus as claimed in claim 9, wherein the fine adjustment image processing means includes means for carrying out the fine adjustment in an interactive manner.

12. Apparatus as claimed in claim 10, wherein the pixel by pixel analysis means includes means for discarding pixels using color variation.

13. Apparatus as claimed in claim 1, further comprising means for recording calibration data obtained during a set up period when no occlusion is present.

14. Apparatus as claimed in claim 1 further comprising a dynamic billboard memory for recording any changes in measured position of the billboard as the camera pans or tilts.

15. An apparatus for automatic measurement of camera orientation, comprising:

(i) a sensor means for measuring a Field of View (FOV) of a camera relative to a known reference position;

(ii) an image processing means for processing video signals generated by the camera; and

(iii) a calibration means for periodically calibrating the sensor means, wherein the calibration means perspectively transforms a current camera image to generate an estminated transformed model, compares a stored image with the estimated transformed model to form a residual video field, and determines residual distortion for calibration by determining residual distortion between the estimated transformed model and the residual video field.

16. A method of automatic replacement of a real billboard in a video image said method comprising the steps of:

recording the Field of View (FOV) parameters of a TV camera in a first reference position by means of motion sensors attached to the camera;

storing a position of the real billboard in a billboard store;

storing a position of the camera sensed by the motion sensors in a camera position store;

determining from the camera position store and the stored position in the billboard store and independently of a video signal generated by the camera whether the real billboard is within the field of view of the camera; and

replacing the real billboard with a replacement billboard when the processor determines that the real billboard is within the field of view of the camera.

17. The method as claimed in claim 16, wherein storing a position of the camera comprises measuring the pan, tilt, zoom or focus of the camera with the motion sensors attached to the camera.

18. The method of claim 16 wherein the billboard comprises a chroma-key panel and in which replacement of the real billboard with the replacement billboard includes the step of occluding said replacement billboard by any object occluding said real billboard.

19. The method of claim 16 further comprising periodically automatically calibrating the motion sensors by perspectively transforming the current video image to provide an estimated transformed model, storing a replica of the billboard image, comparing the perspectively transformed video image with the stored replica to provide a residual video field, solving residual distortion between the transformed model and the residual video field and updating the estimated transformation by means of the residual distortion to provide a calibration correction factor.

20. The method of claim 16, wherein said step of solving the residual distortion includes analysis based in the spatial derivatives of the current image of the transformed model and analyzing pixel by pixel the image differences of the transformed model and the current image.

21. The method of claim 16, wherein automatically calibrating the motion sensors further comprises interactively calibrating the motion sensors.

22. The method of claim 20, wherein the pixel by pixel analysis step includes discarding pixels by using color variation.

23. The method of claim 16, wherein the step of recording calibration data is carried out during a set up period when no occlusion is present.

24. The method of claim 16, further comprising calibrating a difference signal for recalibration of the motion sensors when the present video image is above a predetermined quality factor.

25. The method of claim 24, wherein the replacement billboard is positioned at the last recalibrated position in the event that a recalibration is not possible due to the present video image being below the predetermined quality factor.