Telestrator system

• US 7,492,363 B2
• Filed: 12/08/2005
• Issued: 02/17/2009
• Est. Priority Date: 10/21/1999
• Status: Expired due to Term

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportsvision, Inc.
• Inventors: Hsiao, Walter, White, Marvin S., Gloudemans, James R., Cavallaro, Richard H., Honey, Stanley K., Meier, Kevin R.
• Primary Examiner(s): Nguyen; Phu K
• Application Number:
  US11/297,036
• Publication Number:
  US 20060087504A1
• Time in Patent Office: 1,167 Days
• Field of Search: 345/629, 345/630, 345/589, 345/634, 345/419, 345/421, 345/422, 358/2.1, 358/2.99, 382/276, 382/284
• US Class Current: 345/419
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A telestrator system is disclosed that allows a broadcaster to annotate video during or after an event. For example, while televising a sporting event, an announcer (or other user) can use the present invention to draw over the video of the event to highlight one or more actions, features, etc. In one embodiment, when the announcer draws over the video, it appears that the announcer is drawing on the field or location of the event. Such an appearance can be performed by mapping the pixels location from the user'"'"'s drawing to three dimensional locations at the event. Other embodiments include drawing on the video without obscuring persons and/or other specified objects, and/or smoothing the drawings in real time.

Claims

1. A method of annotating video, comprising:

providing one or more exclusions for blending video images;

receiving video, at least a portion of said video depicting a surface at a live event and a set of one or more objects occluding at least a portion of said surface;

receiving a graphic manually created by a human operator during said live event; and

blending said graphic with said at least a portion of said video using said one or more exclusions to avoid drawing said graphic over at least a portion of said set of one or more objects occluding said at least a portion of said surface.

2. The method of claim 1, wherein providing one or more exclusions for blending video images comprises:

providing a color range that should not be enhanced during said blending said graphic.

3. The method of claim 2, wherein:

providing a color range comprises providing a color map including blending values for a set of colors, said color map including a first blending value for a first color that designates no blending;

said one or more objects occluding at least a portion of said surface include said first color; and

blending said graphic includes;

comparing a first color value for a first pixel in said at least a portion of said video to said color map,

identifying said first blending value for said first color based on said comparing of said first color value, and

using said blending value so as not to blend a corresponding pixel in said graphic with said first pixel in said least a portion of said video.

4. The method of claim 1, further comprising:

accessing color data for a first portion of a first video image from said video;

wherein blending said graphic with said video includes using said color data and said one or more exclusions to cause said graphic to be blended with said first portion of said first video image and not other portions of said first video image.

5. The method of claim 1, wherein blending said graphic includes:

adding said graphic to a first video image at a first position in said first video image, said first position corresponding to a position of a first portion of said surface in said first video image; and

adding said graphic to a second video image at a second position in said second video image, said second position corresponding to a position of said first portion of said surface in said second video image, said first position is different than said second position.

6. The method of claim 5, wherein receiving video includes:

receiving said first video image from a first camera; and

receiving said second video image from a second camera.

7. The method of claim 1, wherein receiving said graphic includes receiving two-dimensional position information for said graphic created in relation to a two-dimensional image, said method further comprising:

converting a first set of one or more two-dimensional positions corresponding to said two-dimensional information to one or more real-world locations; and

converting said one or more real-world locations to a second set of one or more two-dimensional positions;

wherein blending said graphic includes blending said graphic with said at least a portion of said video based on said second set of one or more two-dimensional positions.

8. A method of annotating video, comprising:

storing a blending value corresponding to a particular color in a set of colors;

receiving video depicting a surface at a live event, said surface including said particular color;

receiving a graphic manually created by a human operator during said live event; and

blending said graphic with said video using said blending value corresponding to said particular color such that said graphic appears to be drawn on said surface.

9. The method of claim 8, further comprising:

storing a color map for said set of colors, said color map includes said blending value corresponding to said particular color;

wherein blending said graphic includes;

comparing a first color value for a first pixel in said video to said color map, said first color value corresponding to said particular color,

identifying said blending value for said particular color based on said comparing, and

using said blending value so as to blend a corresponding pixel in said graphic with said first pixel in said video.

10. The method of claim 8, further comprising:

accessing color data for a first portion of a first video image from said video;

wherein blending said graphic with said video includes causing said graphic to be blended with said first portion of said first video image and not other portions of said first video image.

11. The method of claim 10, wherein:

said first portion of said first video image depicts a first portion of said surface;

said accessing color data includes accessing color data for a second portion of said first video image, said second portion of said first video image depicting one or more objects occluding a second portion of said surface, said one or more objects including said particular color; and

blending said graphic includes using said blending value such that said graphic is added to said first portion of said first video image and not said second portion of said first video image.

12. The method of claim 10, wherein:

said first portion of said first video image depicts a first portion of said surface, said first portion of said surface including said particular color;

said accessing color data includes accessing color data for a second portion of said first video image, said second portion depicting one or more objects occluding a second portion of said surface, said one or more objects including a different color in said set of colors; and

blending said graphic includes using said blending value such that said graphic is added to said first portion of said first video image and not said second portion of said first video image.

13. The method of claim 8, wherein receiving said graphic includes receiving two-dimensional position information for said graphic created in relation to a two-dimensional image, said method further comprising:

converting a first set of one or more two-dimensional positions corresponding to said two-dimensional information to one or more real-world locations; and

converting said one or more real-world locations to a second set of one or more two-dimensional positions;

wherein blending said graphic includes blending said graphic with said at least a portion of said video based on said second set of one or more two-dimensional positions.

14. A method of annotating video, comprising:

receiving video depicting a surface at a live event;

receiving two-dimensional position information for at least a portion of a graphic created in relation to a two-dimensional image;

converting a first set of one or more two-dimensional positions corresponding to said two-dimensional information to one or more real-world locations;

converting said one or more real-world locations to a second set of one or more two-dimensional positions; and

blending said graphic with said video based on said second set of one or more two-dimensional positions.

15. The method of claim 14, further comprising:

receiving camera sensor data for a first camera, said two-dimensional image and said video being captured by said first camera;

wherein said first set of one or more two-dimensional positions is in a coordinate system defined relative to said first camera, said converting said first set of one or more two-dimensional positions to one or more real-world locations uses said camera sensor data;

wherein said second set of one or more two-dimensional positions is in said coordinate system, said converting said one or more real-world locations to said second set of one or more two-dimensional positions uses said camera sensor data.

16. The method of claim 14, further comprising:

receiving camera sensor data for a first camera, said video being captured by said first camera;

receiving camera sensor data for a second camera, said two-dimensional image being captured by said second camera;

wherein said first set of one or more two-dimensional positions is in a coordinate system defined relative to said second camera, said converting said first set of one or more two-dimensional positions to said one or more real-world locations uses said camera sensor data for said second camera;

wherein said second set of one or more two-dimensional positions is in a coordinate system defined relative to said first camera, said converting said one or more real-world locations to said second set of one or more two-dimensional positions uses said camera sensor data for said first camera.

17. The method of claim 14, further comprising:

creating real-world model of at least a portion of said surface, said converting said first set of one or more two-dimensional positions to said one or more real-world locations includes using said model.

18. A method of annotating video, comprising:

receiving from a first camera first video depicting a surface at a live event;

receiving from a second camera second video depicting said surface at said live event;

receiving position information for at least a portion of a graphic created in relation to a two-dimensional image of said first video depicting said surface;

converting a first set of one or more positions corresponding to said position information to one or more real-world locations;

converting said one or more real-world locations to a second set of one or more positions; and

blending said graphic with said second video using said second set of one or more two-dimensional positions.

19. The method of claim 18, wherein:

prior to said blending, at least a portion of said second video depicts at least a portion of said surface at said live event and one or more objects occluding said surface; and

said blending includes blending said graphic with said at least a portion of said video so as to avoid drawing said graphic over said one or more objects.

20. The method of claim 19, wherein:

said method further comprises creating a real-world model of at least a portion of said surface;

said first set of one or more positions is in a coordinate system defined relative to said first camera;

said converting said first set of one or more positions to said one or more real-world locations includes using said model;

said second set of one or more positions is in a coordinate system defined relative to said second camera; and

said converting said second set of one or more positions to said one or more real-world locations includes using said model.