System for enhancing a video presentation of a live event

• US 20010005218A1
• Filed: 01/26/2001
• Published: 06/28/2001
• Est. Priority Date: 09/04/1998
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: Gepner, Jerry N., OʼBrien, Terence J., Hsiao, Walter, White, Marvin S., Gloudemans, James R., Cavallaro, Richard H., Honey, Stanley K.
• Granted Patent: US 6,597,406 B2
• Time in Patent Office: Days
• US Class Current: 348/157
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/2723 Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

A three-dimensional model is created to represent an environment to be captured on video. A camera is fitted with pan, tilt and/or zoom sensors. An operator selects a location in the environment. The three-dimensional model is used to determine the three-dimensional coordinates of the location selected by the operator. Information from the pan, tilt and/or zoom sensors is used to transform the three-dimensional coordinates to a two-dimensional position in the video from the camera. Using the two-dimensional position of the video, a graphic is properly added to the video such that the graphic appears to be at the selected location in the environment.

Claims

1. A method of enhancing video, comprising:

selecting a location;

determining a set of coordinates of said location based on a three dimensional model of at least a portion of an environment, said portion of said environment including said location;

converting said set of coordinates of said location to position data representing position in said video; and

enhancing said video based on said position data.

2. A method according to

claim 1

, wherein

said step of selecting includes selecting a location on a field;

said step of determining a set of coordinates includes determining a first subset of points and a second subset of points at said location, said first set of points represents a first border for a line graphic, said second subset of points represents a second border of said line graphic;

said step of converting said set of coordinates converts said first subset of points to a first set of positions and said second subset of points to a second set of positions; and

said step of enhancing includes adding said line graphic to said video based on said first set of positions and said second set of positions.

3. A method according to

claim 1

, wherein:

said step of selecting includes selecting a location of a vertex of a rectangle bounding a logo;

said step of determining a set of coordinates includes determining a set of sub-rectangles and determining three dimensional coordinates of vertices of said sub-rectangles;

said step of converting said set of coordinates converts said three dimensional coordinates of said vertices of said sub-rectangles a set of positions; and

said step of enhancing includes adding said logo to said video based on said set of positions.

4. A method according to

claim 1

, wherein:

said step of selecting includes using a keyboard to type in a yard indicator.

5. A method according to

claim 1

, wherein:

said step of selecting includes using a mouse to point to said location.

6. A method according to

claim 1

, wherein:

said location is a wall on a stadium.

7. A method according to

claim 1

, wherein:

said location is space between goal posts.

8. A method according to

claim 1

, wherein:

said step of enhancing includes keying a graphic of a line over a frame of video captured by a camera.

9. A method according to

claim 1

, wherein:

said step of enhancing includes editing a frame of video.

10. A method according to

claim 1

, wherein:

said step of converting includes multiplying said coordinates by a transformation matrix.

11. A method according to

claim 10

, wherein said step of determining a set of coordinates includes:

determining whether said models includes information for said location;

interpolating to determine said set of coordinates if said model does not include said information for said location.

12. A method according to

claim 1

, farther including the step of:

creating said model.

13. A method according to

claim 1

, wherein:

said step of enhancing adds a line to said video at a first position based on said position data; and

said method further including the step of manually adjusting said first position.

14. A method according to

claim 1

, wherein:

said step of determining does not utilize pattern recognition.

15. A method according to

claim 1

, further including the step of

receiving field of view data for a first camera, said step of converting is based on said field of view data, said video is captured by said first camera.

16. A method according to

claim 15

, further including the step of:

determining whether said first camera is tallied based on comparing said video captured by said camera to a tallied video signal, said step of enhancing enhances said video if said first camera is tallied.

17. A method according to

claim 1

, further including the step of:

creating inclusion criteria defining at least two colors of inclusion, said step enhancing includes editing pixels that meet said inclusion criteria.

18. A method according to

claim 1

, further including the steps of creating inclusion criteria defining at least two colors of inclusion, said step enhancing includes editing pixels that meet said inclusion criteria;

receiving field of view data for a first camera, said step of converting is based on said field of view data, said video is captured by said first camera; and

determining whether said first camera is tallied based on comparing said video captured by said camera to a tallied video signal, said step of enhancing enhances said video if said first camera is tallied.

19. One or more processor readable storage devices having processor readable code embodied on said processor readable storage medium, said processor readable code for programming a processor to perform a method comprising the steps of:

receiving a selecting of a location;

determining a set of coordinates of said location based on a three dimensional model of at least a portion of an environment, said portion of said environment including said location;

converting said set of coordinates of said location to position data representing position in said video; and

directing an enhancement to said video based on said position data.

20. One or more processor readable storage devices according to

claim 19

, wherein:

said step of directing an enhancement directs the addition of a line to said video at a first position based on said position data.

21. One or more processor readable storage devices according to

claim 19

, wherein said method further includes the step of:

receiving field of view data for a first camera, said step of converting is based on said field of view data, said video is captured by said first camera.

22. One or more processor readable storage devices according to

claim 19

, wherein said method further includes the step of:

determining whether said first camera is tallied based on comparing said video captured by said camera to a tallied video signal, said step of enhancing enhances said video if said first camera is tallied.

23. One or more processor readable storage devices according to

claim 19

, wherein said method further includes the step of:

creating inclusion criteria defining at least two colors of inclusion, said step directing an enhancement includes causing the editing of pixels that meet said inclusion criteria.

24. An apparatus for enhancing video captured by a first camera, comprising:

a first set one or more field of view sensors adapted to sense field of view information from said first camera;

a processor in communication with said first set of one or more field of view sensors;

a processor readable storage unit in communication with said processor, said processor readable storage unit storing processor readable code for programming said processor to perform a method comprising the steps of;

receiving a selecting of a location,

determining a set of coordinates of said location based on a three dimensional model of at least a portion of an environment, said portion of said environment including said location,

converting said set of coordinates of said location to position data representing position in said video, and

directing an enhancement to said video based on said position data.

25. An apparatus according to

claim 24

, further including:

video modification hardware in communication with processor, said video modification hardware enhances said video.

26. An apparatus according to

claim 24

, further including:

said video modification hardware includes a keyer.

27. An apparatus according to

claim 24

, further including:

a tally detector in communication with said processor.

28. An apparatus according to

claim 24

, further including:

a second camera;

a second set of one or more field of view sensors for said second camera, said second set of one or more field of view sensors in communication with said processor;

a third camera;

a third set of one or more field of view sensors for said third camera, said third set of one or more field of view sensors in communication with said processor; and

a tally detector in communication with said processor, said first camera, said second camera and said third camera.

29. A method of enhancing video, comprising:

manually selecting a location to be highlighted;

determining said location'"'"'s position in said video without using pattern recognition; and

enhancing said video based on said location'"'"'s position.

30. A method according to

claim 29

, wherein:

said step of enhancing adds a line to said video at said location'"'"'s position in said video; and

said method further including the step of manually adjusting said first position.

31. A method according to

claim 29

, further including the step of:

receiving field of view data for a first camera, said step of determining is based on said field of view data, said video is captured by said first camera.

32. A method according to

claim 31

, further including the step of:

determining whether said first camera is tallied based on comparing said video captured by said camera to a tallied video signal, said step of enhancing enhances said video if said first camera is tallied.

33. A method according to

claim 29

, further including the step of:

creating inclusion criteria defining at least two colors of inclusion, said step enhancing includes editing pixels that meet said inclusion criteria.

34. A method of enhancing video, comprising:

capturing first video using a first camera;

determining whether said first camera is tallied based on comparing said first video to a tallied video signal; and

enhancing said first video if said camera is tallied.

35. A method according to

claim 34

, further including the steps of:

capturing second video using a second camera;

determining whether said second camera is tallied based on comparing said second video to said tallied video signal;

enhancing said second video if said camera is tallied;

capturing third video using a third camera;

determining whether said third camera is tallied based on comparing said third video to a tallied video signal; and

enhancing said third video if said camera is tallied.

36. A method according to

claim 35

, further including the steps of:

selecting a location to be highlighted; and

determining said location'"'"'s position in said first video if said first camera is tallied;

determining said location'"'"'s position in said second video if said second camera is tallied; and

determining said location'"'"'s position in said third video if said third camera is tallied, said step of enhancing is based on said location'"'"'s position in said first video if said first camera is tallied, said step of enhancing is based on said location'"'"'s position in said first video if said first camera is tallied and said step of enhancing is based on said location'"'"'s position in said third video if said third camera is tallied.

37. An apparatus for enhancing video captured by a first camera, comprising:

a tally detector receiving a video signal from said first camera and a tallied video signal, said tally detector compares said signal from said first camera to said tallied video signal and reports whether said first camera is tallied;

a processor in communication with said tally detector;

a processor readable storage unit in communication with said processor, said processor readable storage unit storing processor readable code for programming said processor to enhancing said video if said first camera is tallied.

38. An apparatus according to

claim 37

, further including:

a second camera in communication with said tally detector; and

a third camera in communication with said tally detector.

39. An apparatus according to

claim 37

, further including:

a pan sensor for said first camera; and

a tilt sensor for said second camera, said pan sensor and said tilt sensor being in communication with said processor, said processor receives an indication of a location in three dimensional space and converts said location to a position in said video based on information from said pan sensor and said tilt sensor, said processor enhances said video based on said position in said video.

40. A method of enhancing video, comprising:

creating inclusion criteria defining at least two colors of inclusion;

selecting a location to be highlighted;

determining said location'"'"'s position in said video during a live event; and

enhancing said video at said location'"'"'s position in said video, said step of enhancing includes editing pixels that meet said inclusion criteria.

41. A method according to

claim 40

, wherein:

said inclusion criteria includes one or more display filters.

42. A method according to

claim 41

, wherein:

said display filters include criteria for luminance and chrominance.

43. A method according to

claim 40

, wherein:

said step of creating inclusion criteria includes selecting a portion of a frame of video.

44. A method according to

claim 40

, further including the step of:

creating exclusion criteria, said step of enhancing does not edit pixels meeting said exclusion criteria.

45. A method according to

claim 44

, wherein:

said step of enhancing adds a line to said video at said locations position in said video;

said inclusion criteria identifies a color of a field; and

said exclusion criteria identifies a color of a player'"'"'s uniform.

46. One or more processor readable storage devices having processor readable code embodied on said processor readable storage medium, said processor readable code for programming a processor to perform a method comprising the steps of:

creating one or more display filters defining at least two colors of inclusion;

receiving a selection of a location to be highlighted;

determining said location'"'"'s position in said video during a live event; and

directing an enhancement to said video at said location'"'"'s position in said video, said step of enhancing includes editing pixels that meet said inclusion criteria.

47. One or more processor readable storage devices according to

claim 46

, wherein said method further includes the step of:

creating exclusion criteria, said step of enhancing does not edit pixels meeting said exclusion criteria.

48. One or more processor readable storage devices according to

claim 47

, wherein:

said step of enhancing adds a line to said video at said locations position in said video;

said inclusion criteria identifies a color of a field; and

said exclusion criteria identifies a color of a player'"'"'s uniform.

49. An apparatus for enhancing video captured by a first camera, comprising:

one or more field of view sensors adapted to sense field of view information from said first camera;

a processor in communication with said one or more field of view sensors, said processor programmed to perform a method comprising the steps of;

creating one or more display filters defining at least two colors of inclusion,

receiving a selection of a location to be highlighted, and

determining said location'"'"'s position in said video during a live event; and

video modification hardware in communication with said processor, for enhancing said video at said location'"'"'s position in said video, said step of enhancing includes editing pixels that meet said inclusion criteria.

50. An apparatus according to

claim 49

, wherein said method further includes the step of:

creating exclusion criteria, said step of enhancing does not edit pixels meeting said exclusion criteria.

51. An apparatus according to

claim 50

, wherein:

said step of enhancing adds a line to said video at said locations position in said video;

said inclusion criteria identifies a color of a field; and

said exclusion criteria identifies a color of a player'"'"'s uniform.

52. A method of enhancing video, comprising:

determining field of view information for a camera capturing said video;

selecting a location on a field to add a yardline;

determining a set of coordinates of said location based on a three dimensional model of at least a portion of said field;

converting said set of coordinates of said location to position data for said video, said step of converting being based on said field of view data; and

adding a graphic of a yardline to said video based on said position data.

53. A method according to

claim 52

, further including the step of:

determining whether said camera is tallied based on comparing said video captured by said camera to a tallied video signal, said step of adding a graphic adds said graphic of a yard line if said first camera is tallied.

54. A method according to

claim 52

, further including the step of:

creating inclusion criteria defining at least two colors of inclusion, said step enhancing includes editing pixels that meet said inclusion criteria.

55. A method according to

claim 52

, further including the step of:

creating exclusion criteria, said step of enhancing does not edit pixels meeting said exclusion criteria.