System for enhancing video

• US 7,154,540 B2
• Filed: 04/27/2001
• Issued: 12/26/2006
• Est. Priority Date: 10/22/1996
• Status: Expired due to Fees

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Fox Sports Productions, Inc. (The Walt Disney Company)
• Inventors: Hill, David Blyth, Goren, Edward Gerald, Gepner, Jerry Neil, Cavallaro, Richard H., Honey, Stanley K.
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): Hannett, James M.
• Application Number:
  US09/844,524
• Publication Number:
  US 20010026319A1
• Time in Patent Office: 2,069 Days
• Field of Search: 348/578, 348/601, 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

Pan, tilt and zoom sensors are coupled to a broadcast camera in order to determine the field of view of the broadcast camera and to make a rough estimate of a target'"'"'s location in the broadcast camera'"'"'s field of view. Pattern recognition techniques can be used to determine the exact location of the target in the broadcast camera'"'"'s field of view. If a preselected target is at least partially within the field of view of the broadcast camera, all or part of the target'"'"'s image is enhanced. The enhancements include replacing the target image with a second image, overlaying the target image or highlighting the target image. Examples of a target include a billboard, a portion of a playing field or another location at a live event. The enhancements made to the target'"'"'s image can be seen by the television viewer but are not visible to persons at the live event.

Claims

1. A method for enhancing the broadcast of a target at a live event, comprising the steps of:

capturing a first frame of video using a first camera;

capturing a second frame of video using a second camera, said second frame of video including said target;

determining if said target is within said first frame of video;

determining a position and orientation of said target in said first frame of video;

detecting an occlusion of said target in said second frame of video;

determining where said detected occlusion is positioned in said first frame of video at least partially based on said step of detecting; and

modifying said first frame of video by enhancing said target in said first frame of video without enhancing said detected occlusion.

2. A method according to claim 1, wherein:

said second camera is pointed at said target and is located substantially adjacent said first camera;

said step of detecting an occlusion includes comparing at least a portion of said second frame of video to an unoccluded image of said target.

3. A method according to claim 2, wherein:

said second camera is zoomed such that said target fills a substantial portion of said second frame of video.

4. A method for adding a graphic indication of a first down to a broadcast of a football game during said broadcast of said football game, comprising:

storing an indication of a location on a football field, said location corresponding to said first down;

sensing field of view data for a camera using one or more field of view sensors that do not use image pattern recognition;

receiving video from said camera; and

adding a line to said video using image pattern recognition and said one or more field of view sensors that do not use image pattern recognition, said adding said line includes determining said location'"'"'s position in said video using said field of view data and inserting said line into said video at said location'"'"'s position in said video, said step of inserting includes determining whether said video includes an occlusion of said location'"'"'s position using image pattern recognition, said step of inserting includes adding said line to one or more portions of said video that are not occluded and not adding said line to one or more portions of said video that are occluded.

5. An apparatus for adding a graphic indication of a first down to a broadcast of a football game during said broadcast of said football game, comprising:

one or more field of view sensors associated with a camera, said one or more field of view sensor do not use image pattern recognition;

a storage device; and

one or more processing units in communication with said storage device and said one or more field of view sensors, said storage device stores data for said one or more processing units, said one or more processing units capable of performing a method comprising;

storing an indication of a location on a football field, said location corresponding to said first down,

receiving field of view data from said one or more field of view sensors;

receiving video from said camera, and

adding a line to said video using image pattern recognition and said one or more field of view sensors that do not use image pattern recognition, said adding said line includes determining said location'"'"'s position in said video using said field of view data and inserting said line into said video at said location'"'"'s position in said video, said step of inserting includes determining whether said video includes an occlusion of said location'"'"'s position using image pattern recognition, said step of inserting includes adding said line to one or more portions of said video that are not occluded and not adding said line to one or more portions of said video that are occluded.

6. A method for adding a graphic indication of a first down to a broadcast of a football game during said broadcast of said football game, comprising the steps of:

storing an indication of a location on a football field, said location corresponding to said first down;

sensing field of view data for a camera using one or more field of view sensors that do not use image pattern recognition;

receiving video from said camera;

determining said location'"'"'s position in said video using a first step process that makes use of said field of view data;

performing a second step process to refine said position'"'"'s location in said video, said second step process uses image pattern recognition;

adding a line to said video at said location'"'"'s position in said video.