Method and apparatus for enhancing the broadcast of a live event

• US 20010026319A1
• Filed: 04/27/2001
• Published: 10/04/2001
• Est. Priority Date: 10/22/1996
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Fox Sports Productions, Inc. (The Walt Disney Company)
• Inventors: Cavallaro, Richard H., Honey, Stanley K., Hill, David Blyth, Goren, Edward Gerald, Gepner, Jerry Neil
• Granted Patent: US 7,154,540 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

Pan, tilt and zoom sensors are coupled to a broadcast camera in order to determine the field of view of the broadcast camera and to make a rough estimate of a target'"'"'s location in the broadcast camera'"'"'s field of view. Pattern recognition techniques can be used to determine the exact location of the target in the broadcast camera'"'"'s field of view. If a preselected target is at least partially within the field of view of the broadcast camera, all or part of the target'"'"'s image is enhanced. The enhancements include replacing the target image with a second image, overlaying the target image or highlighting the target image. Examples of a target include a billboard, a portion of a playing field or another location at a live event. The enhancements made to the target'"'"'s image can be seen by the television viewer but are not visible to persons at the live event.

Claims

1. A method for enhancing the broadcast of a live event, comprising the steps of:

capturing first video using a first camera;

sensing field of view data representing a field of view of said first camera;

determining a position and orientation of a video image of a target in said captured video at least partially based on recognizing one or more portions of said video image of said target in said captured video; and

modifying said captured video data by enhancing at least a segment of said video image of said target.

2. The method according to

claim 1

, wherein said step of determining a position includes the steps of:

determining a rough estimate of said position of said target in said captured video using said field of view data; and

determining a more precise estimate of said position of said target in said captured video using a pattern recognition technique.

3. The method according to

claim 1

, further including the step of:

determining whether said target is within said field of view of said first camera.

4. The method according to

claim 1

, wherein:

the step of determining is also at least partially based on comparing said field of view data to prestored location data for said target.

5. The method according to

claim 1

, wherein:

said step of modifying replaces a first advertisement with a second advertisement.

6. The method according to

claim 1

, wherein:

said step of modifying replaces an image of a surface in a stadium with an advertisement.

7. The method according to

claim 1

, wherein:

said step of modifying includes highlighting a portion of a playing field.

8. The method according to

claim 1

, wherein:

enhancing said video image of said target does not include replacing said video image of said target; and

said method further including the step of accounting for occlusions.

9. The method according to

claim 1

, further including the steps of:

capturing second video using a second camera, said second video including said target, said second camera zoomed such that said target substantially fills most of said second camera'"'"'s field of view;

detecting an occlusion of said target in said second video; and

using said detection of said occlusion from said second video to determine where said occlusion is positioned in said first video;

said step of modifying said first video does not replace said occlusion.

10. The method according to

claim 1

, further including the steps of:

storing said target'"'"'s location before said step of capturing; and

storing an unoccluded image of said target before said step of capturing.

11. A method according to

claim 1

, further including the step of:

learning changes to said target image.

12. The method according to

claim 1

, further including the steps of:

comparing said video image of said target in said captured video with a video image stored in a memory; and

updating said memory to include a revised image of said target.

13. A method for enhancing the broadcast of a video image of a target at a live event, comprising the steps of:

capturing a frame of video using a first camera;

sensing an electromagnetic signal transmitted from said target, said electromagnetic signal not being visible to the human eye;

determining a position and orientation of said video image of said target in said frame of video, at least partially based on said electromagnetic signal; and

modifying said video data by enhancing at least a segment of said video image of said target.

14. A method according to

claim 13

, wherein:

said step of determining includes determining the pixel position of the target in said sensor frame of data.

15. A method according to

claim 13

, wherein

said electromagnetic signal is an infrared signal.

16. A method according to

claim 13

, further including the step of:

storing data, based on said electromagnetic signal, that has a value greater than a predetermined threshold.

17. A method according to

claim 13

, further including the step of:

ignoring data from said electromagnetic signal if sensed during a flash.

18. A method for enhancing the broadcast of a target at a live event, comprising the steps of:

capturing a first frame of video using a first camera;

capturing a second frame of video using a second camera, said second frame of video including said target;

determining if said target is within said first frame of video;

determining a position and orientation of said target in said first frame of video;

detecting an occlusion of said target in said second frame of video;

determining where said detected occlusion is positioned in said first frame of video at least partially based on said step of detecting; and

modifying said first frame of video by enhancing said target in said first frame of video without enhancing said detected occlusion.

19. A method according to

claim 18

, wherein:

said second camera is pointed at said target and is located substantially adjacent said first camera;

said step of detecting an occlusion includes comparing at least a portion of said second frame of video to an unoccluded image of said target.

20. A method according to

claim 19

, wherein:

said second camera is zoomed such that said target fills a substantial portion of said second frame of video.

21. A method according to

claim 19

, further including the steps of:

storing said unoccluded image of said target prior to said step of capturing said first frame of video; and

updating said stored unoccluded image of said target if lighting conditions change.

22. A system to be used with a first camera for enhancing the broadcast of a target at a live event, comprising:

one or more field of view sensors coupled to said camera such that said one or more field of view sensors can detect field of view data representing said first camera'"'"'s field of view;

memory storing a location of said target; and

one or more processors, in communication with said memory and said one or more field of view sensors, said one or more processors programmed to determine whether said target is within the field of view of said camera and to determine where said target is positioned within a frame of video of said first camera.

23. A system according to

claim 22

, wherein:

said memory stores data representing a video image of said replacement graphic.

24. A system according to

claim 22

, further including:

a video modification unit, in communication with said one or more processors, for modifying said frame of video to enhance at least a section of said video image of said target with a replacement graphic.

25. A system according to

claim 24

, wherein:

said video modification unit is a linear keyer.

26. A system according to

claim 24

, wherein:

said video modification unit is a processor.

27. A system according to

claim 24

, wherein:

said video modification unit highlight a portion of a football field.

28. A system according to

claim 24

, wherein:

said video modification unit replaces a first billboard with a second billboard.

29. A system according to

claim 24

, wherein:

said video modification unit adds a first billboard to said frame of video.

30. A system according to

claim 22

, wherein:

said one or more field of view sensors includes a pan sensor, a tilt sensor and a zoom sensor.

31. A system according to

claim 22

, further including:

a second camera pointed at said target, in communication with said one or more processors and located substantially adjacent to said first camera.

32. A system according to

claim 22

, further including:

a video control in communication with said first camera and said one or more processors;

a video mixer in communication with said second camera and said one or more processors; and

a video delay unit in communication with said video control and said video modification unit.

33. A system for enhancing the broadcast of target at a live event, comprising:

a plurality of broadcast cameras;

a plurality of field of view sensors, each sensor coupled to one of said broadcast cameras;

a multiplexer in communication with said field of view sensors for selectively transmitting a signal from one of said field of view sensors;

a video delay unit;

a video control unit in communication with said broadcast cameras, said video control unit selectively transmits to said video delay unit a signal from one of said broadcast cameras;

a plurality of dedicated cameras with a fixed field of view and pointed at one of said plurality of targets, each dedicated camera located substantially adjacent to a broadcast camera;

a video mixer in communication with said video control unit and said dedicated camera for selectively transmitting a signal from one of said dedicated cameras, said selected one of said dedicated cameras being substantially adjacent to said selected one of said broadcast cameras;

memory storing the location of said targets;

one or more processors, in communication with said memory and said multiplexer, said one or more processors receives said selected signal from said video control unit, said one or more processors programmed to determine whether one of said targets is positioned within the field of view of one of said broadcast cameras and to determine where said one target is within a frame of video of said one broadcast camera;

a video modification unit, in communication with said one or more processors, for modifying said frame of video to enhance at least a section of said video image of said target with a replacement graphic.

34. A system, to be used with a first camera, for enhancing the broadcast of a live event, comprising:

a target including an electromagnetic transmitter;

a sensor adapted to receive an electromagnetic signal from said target,

said electromagnetic signal is not visible to a human eye;

a memory storing the location of said target;

one or more processors, in communication with said memory and said sensor, said one or more processors programmed to determine whether said target is within the field of view of said first camera and to determine where said target is within a frame of video of said first camera.

35. A system according to

claim 34

, further including:

a video modification unit, in communication with said one or more processors, for modifying said frame of video to replace at least a section of said video image of said target with at least a replacement graphic.

36. A system according to

claim 34

, wherein:

said electromagnetic signal is an infrared signal.