Method and system for measurement of the duration an area is included in an image stream

• US 7,206,434 B2
• Filed: 07/10/2001
• Issued: 04/17/2007
• Est. Priority Date: 07/10/2001
• Status: Expired due to Term

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Vistas Unlimited, Inc. (The Walt Disney Company)
• Inventors: Hayden, Charles H. Jr., Carmichael, Norman S., Muench, Mark S., Overton, Kenneth J.
• Primary Examiner(s): Mehta; Bhavesh M.
• Assistant Examiner(s): LAVIN, CHRISTOPHER L
• Application Number:
  US09/903,122
• Publication Number:
  US 20030012409A1
• Time in Patent Office: 2,107 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes

Abstract

A system and method for calculation of the duration a target area included within an image stream utilizes a three-dimensional model of at least one target area within a site. A counter is incremented during periods, for example on a frame-by-frame basis, when the target area is included in the image stream.

An occlusion parameter is calculated for adjusting the calculated duration that the target area is included within the image stream. The occlusion parameter facilitates accounting for variations in a target area obstruction and is used for making adjustments to a counter so that a count indicative of a duration reflects variations in obstructions of the target area. A foreground parameter associated with target area is calculated for adjusting the calculated duration of the target area. The foreground parameter facilitates accounting for variations in the viewability of an image included within the target area resulting from various camera zooms.

Claims

1. A method of calculating the duration of a target area is included within an image stream obtained by an image capture system at a physical site, the method comprising:

analyzing at least one field of the image stream for inclusion of the target area, wherein the target area is defined in a three-dimensional computer model of the site, the three-dimensional computer model being defined independently of a viewpoint of a camera generating the image stream, and the target area being located within the field using information on where the camera is positioned and pointing and the three-dimensional computer model;

automatically incrementing a counter upon confirmation that the target area is included within the at least one field; and

calculating an occlusion parameter of the target area;

wherein the counter is incremented by the numerical result of the occlusion parameter subtracted from one.

2. The method according to claim 1, wherein a fee is calculated in response to the counter.

3. The method according to claim 1, wherein the occlusion parameter is calculated in response to a pixel count of an occlusion of the target area and a pixel count of the target area.

4. The method according to claim 3, wherein the occlusion parameter is a ratio of the pixel count of the occlusion to the pixel count of the target area.

5. The method according to claim 1, further comprising defining an occlusion threshold, an increment to the counter being disallowed if the occlusion threshold exceeds the occlusion parameter.

6. The method according to claim 1, further comprising:

assigning image data for insertion into the target area; and

reassigning the image data to a second target area to be included in the image stream.

7. The method according to claim 6, further comprising calculating the duration the image data is included in the image steam by summing respective increments to the counter that is associated with the first target area with increments to a second counter that is associated with the second target area, the increments summed limited to respective increments made to the target area when the image data is respectively assigned to one or more of the target area and the second target area.

8. The method according to claim 6, wherein reassigning the image data to a second target area further comprises reassigning the image data to the second target area as the image stream is captured.

9. The method according to claim 6, further comprising:

specifying a duration the image data is to be included within the image stream prior to capture of the image stream; and

collecting, prior to reassigning the image data, statistical data indicative of the duration that at least one of the group consisting of the target area and the second target area is included in the image stream as the image stream is captured.

10. The method of claim 1, wherein the target corresponds to an imaginary surface defined in the three-dimensional computer model and does not correspond with a physical element at the site.

11. A method of calculating the duration a target area is included within an image stream obtained by an image capture system, the method comprising:

analyzing at least one field of the image stream for inclusion of the target area, wherein the target area is defined in a three-dimensional computer model of the site, the three-dimensional computer model being defined independently of a viewpoint of a camera generating the image stream, and the target area being located within the field using information on where the camera is positioned and pointing and the three-dimensional computer model;

automatically incrementing a counter upon confirmation that the target area is included within the at least one field; and

calculating a foreground parameter of the target area, the foreground parameter compensating for a zoom of the image capture system;

wherein the foreground parameter is calculated in response to a pixel count of the target area and a pixel count of the at least one field.

12. The method according to claim 11, wherein the foreground parameter is a ratio of the pixel count of the target area to the pixel count of the at least one field.

13. A method of calculating the duration a target area is included within an image stream obtained by an image capture system at a physical site, the method comprising:

analyzing at least one field of the image stream for inclusion of the target area, wherein the target area is defined in a three-dimensional computer model of the site, the three-dimensional computer model being defined independently of a viewpoint of a camera generating the image stream, and the target area being located within the field using information on where the camera is positioned and pointing and the three-dimensional computer model;

automatically incrementing a counter upon confirmation that the target area is included within the at least one field;

calculating a foreground parameter of the target area;

wherein the counter is incremented by an increment that is equivalent to the calculated value of the foreground parameter.

14. A method of calculating the duration a target area is included within an image stream obtained by an image capture system, the method comprising:

analyzing at least one field of the image stream for inclusion of the target area;

automatically incrementing a counter upon confirmation that the target area is included within the at least one field;

assigning image data for insertion into the target area;

specifying a duration the image data is to be included within the image stream prior to capture of the image stream;

reassigning the image data to a second target area to be included in the image stream; and

collecting, prior to reassigning the image data, statistical data indicative of the duration that at least one of the group consisting of the target area and the second target area is included in the image stream as the image stream is captured;

wherein reassigning the image data to a second target area further comprises reassigning the image data to the second target area in response to the statistical data indicating the second target area has a higher duration of inclusion in the image stream than the target area.

15. A system for calculating the duration a target area is included in an image stream of a physical site, the system comprising:

a three-dimensional computer model of a target area from a site from which the image stream is captured, the target area being virtually defined by the model and not corresponding to a physical element at the site;

the three-dimensional computer model being defined independently of a viewpoint of a camera producing the image stream, the target area being located within the field using information on where the camera is positioned and is pointing and the three-dimensional computer model;

a duration calculation module that identifies inclusion of the target area in the at least one field of the image stream and calculates an occlusion parameter for the target area, the module including a counter that is incremented as a function of the occlusion parameter upon confirmation of inclusion of the target area within the at least one field; and

wherein the target area is included in the at least one field when the site is included in the image stream.

16. The system according to claim 15, further comprising:

image insertion system for inserting a target image into at least one of the plurality of fields of the image stream, the image insertion system in communication with the duration calculation module; and

an image capture system for providing an image to the image measurement system, the image supply system in communication with the image insertion system.

17. The system according to claim 15, wherein the duration calculation module is included within the image insertion system.

18. The system according to claim 15, wherein the image measurement system further comprises:

a model renderer for generating a synthetic image based on a predefined three-dimensional reference model of a target area within a site from a known position of a camera included in the image supply system, the synthetic image having the target image inserted in the target area thereof;

an image separator for masking from a video image contained within a frame of a video signal generated by the camera, the target area to create a masked background image, a masked reference image being created by the image separator by separating a reference image from the remainder of the image; and

an occlusion separator operable to compare the masked target area image to the masked reference image, the difference therebetween representing an occlusion of the target image.

19. The system according to claim 18, wherein the module calculates the occlusion parameter from information obtained from the image separator.

20. The system according to claim 18, wherein the module calculates a foreground parameter from information obtained from the image insertion system.

21. The system according to claim 18, wherein the target area comprises image data selected from the group consisting of signage existing in a site from which the image stream was captured and synthetic images not present in the site.

22. The system according to claim 18, further comprising

a model of a target area from a second site from which the image stream is captured;

and

a duration calculation module that identifies inclusion of the target area from the second site in the at least one field of the image stream, the module including a counter that is incremented upon confirmation of inclusion of the target area within the at least one field, and wherein the target area from the second site is included in the at least one field when the second site is included in the image stream.

23. The system according to claim 15, wherein the occlusion parameter is calculated as a ratio of a pixel count of the occlusion to a pixel count of the target image.

24. The system according to claim 20, further comprising an image combiner for producing a final image from to masked background image and the occlusion, the module obtaining a pixel count of the target area from the image separator and a pixel count of the final image from the image combiner, the foreground parameter being calculated from the pixel count of the target area and the pixel count of the final image.

25. The system according to claim 20, wherein the incrementation is a function of the foreground parameter.

26. The system according to claim 15, wherein an increment to the counter is disallowed in the event the occlusion parameter exceeds an occlusion threshold.

27. A method of tracking of appearance of synthetic images inserted into an image stream obtained by an image capture system having a video camera at a physical site, the method comprising:

for each image frame in the image stream,

identifying a target area within the original image frame based on at least a predefined three-dimensional model of a target area within the site and the camera'"'"'s position and pointing direction, the three-dimensional computer model being defined independently of a viewpoint of a camera generating the image stream;

rendering for the target area a synthetic image based at least on the predefined three-dimensional model of the target area;

rendering a mask for separating the target area and background area within the image frame;

rendering an un-occluded reference image for the target area;

separating occlusions within the at least one target area of the original image frame by comparing the target area in the original image frame to the reference image for the target area; and

combining the background of the original image, the synthetic image, and the image of the occlusions into an output image; and

automatically computing a value representative of an appearance of the synthetic image in one of the image frame and a field within the image frame, the value taking into account occlusions of the synthetic image.

28. The method of claim 27, further comprising summing for each frame of the image stream the values representative of the appearance of the synthetic image in one of the image frame and the field within the image frame.

29. The method of claim 27, wherein automatically computing a value representative of appearance of the synthetic image, further comprises automatically computing an occlusion parameter representative of an amount by which the synthetic image in the target area is occluded and adjusting a pre-assigned value representative of the target image appearing in the image as a function of the occlusion parameter.

30. The method of claim 29, further comprising automatically computing a foreground parameter representative of viewability of the synthetic image in the image frame as a function of camera focal length, and adjusting the preassigned value as a function of the foreground parameter.

31. The method of claim 27, wherein the value further takes into account viewability of the synthetic image in the image frame as a function camera focal length.

32. The method of claim 27, wherein the value representative of the appearance of the synthetic is automatically computed based on a predefined portion of the synthetic image.

33. Image processing apparatus comprising:

an image insertion system, the image insertion system including

an identifying mechanism identifying a target area within the original image frame based on at least a predefined three-dimensional model of a target area within the site and a position and pointing direction of a camera generating the image stream, the three-dimensional computer model being defined independently of a viewpoint of a camera generating the image stream;

a mechanism for rendering for the target area a synthetic image and an unoccluded reference image for the target area using the predefined three-dimensional model of the target area and the position and pointing direction of a camera;

a mechanism for rendering a mask for separating the target area and background area within the image frame;

a mechanism for separating occlusions within the at least one target area of the original image frame by comparing the target area in the original image frame to the reference image for the target area; and

a mechanism for combining the background of the original image, the synthetic image, and the image of the occlusions into an output image; and

an image measurement mechanism for automatically computing a value representative of appearance of the synthetic image in one of the image frame and a field within the image frame, the value taking into account occlusions of the synthetic image.

34. The apparatus of claim 33, wherein the image measurement system further comprises a summing mechanism for summing for each frame of the image stream the values representative of the appearance of the synthetic image in one of the image frame and the field within the image frame.

35. The apparatus of claim 34, wherein the image measurement system further comprises a mechanism for automatically computing an occlusion parameter representative of an amount by which the synthetic image in the target area is occluded and adjusting a pre-assigned value representative of the target image appearing in the image as a function of the occlusion parameter.

36. The apparatus of claim 35, further comprising a mechanism for computing a foreground parameter representative of viewability of the synthetic image in the image frame as a function of camera focal length, and adjusting the preassigned value as a function of the foreground parameter.

37. The apparatus of claim 33, wherein the value further takes into account viewability of the synthetic image in the image frame as a function camera focal length.

38. The apparatus of claim 33, wherein the value representative of the appearance of the synthetic image is automatically computed based on a predefined portion of the synthetic image.