Virtual strike zone

• US 7,341,530 B2
• Filed: 12/27/2002
• Issued: 03/11/2008
• Est. Priority Date: 01/09/2002
• Status: Expired due to Term

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: Mozes, Alon, Steinberg, Eric, Gueziec, Andre, Cavallaro, Richard H.
• Primary Examiner(s): PEZZUTO, ROBERT ERIC
• Application Number:
  US10/331,117
• Publication Number:
  US 20030171169A1
• Time in Patent Office: 1,901 Days
• Field of Search: 473/451, 473/455, 473/454, 473/456.Q, 473/462, 473/468, 473/453, 473/499, 473/500, 273/108, 273/118, 273/148
• US Class Current: 473/455
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8011 Ball

Abstract

A system adds a graphical image of the strike zone to a video or other image of a baseball game. The system determines location of the strike zone and the ball in real space. The locations of the strike zone and the ball are depicted in the video. Based on knowing the locations of the strike zone and the ball, the system can determines whether the pitch was a strike or a ball.

Claims

1. A method of providing strike zone information, comprising the steps of:

determining a set of one or more locations of a ball from a first set of video images;

determining a three dimensional volume representing a strike zone;

determining whether one of said one or more locations of said ball intersects the three dimensional volume representing the strike zone;

converting the three dimensional volume and one or more locations of a ball to two-dimensional locations within a second video image using data in the second video image; and

indicating whether one of said set of one or more locations of said ball intersects the three dimensional volume representing the strike zone,

wherein said step of determining a set of one or more locations of a ball includes the steps of;

receiving the first set of video images that depict said ball,

searching for data in a restricted area of said video images, said restricted area being an area where said ball is expected to be during a pitch, and

using said data to automatically determine said one or more locations.

2. A method according to claim 1, further comprising the steps of:

determining a location of said strike zone;

determining a first position in a video, said first position corresponding to said location of said strike zone;

adding an image for said strike zone to said video at said first position.

3. A method according to claim 1, wherein:

said step of indicating includes displaying crosshairs, said crosshairs have a center, said center represents a position where said ball intersects a front surface of said three dimensional volume representing the strike zone.

4. A method according to claim 1, wherein:

said step of indicating includes displaying a location of said strike zone and a location of said ball.

5. A method according to claim 1, wherein:

said step of indicating includes adding an image of said strike zone to video of a baseball game.

6. A method according to claim 1, wherein said step of converting includes:

transforming the three dimensional volume and one or more locations of a ball to two-dimensional locations within the second video; and

adjusting the positions of the ball and three dimensional volume based on offsets derived from transformed edges based on camera sensor data and actual edges in the second video image.

7. A method according to claim 2, further comprising the steps of:

determining a second position in said video, said second position corresponding to one of said set of one or more locations of said ball;

adding an image for said ball to said video based on said second position; and

determining whether said ball intersects said strike zone.

8. A method according to claim 2, wherein:

said video is from a first camera;

said first camera can change orientation;

said first camera includes one or more camera sensors for measuring orientation information of said first camera; and

said step of determining a first position uses said orientation information.

9. A method according to claim 8, wherein:

said one or more camera sensors includes a shaft encoder to measure pan and a shaft encoder to measure tilt.

10. A method according to claim 8, wherein:

said step of determining a first position includes correcting for error in said orientation information.

11. A method according to claim 8, wherein:

said step of determining a first position includes correcting for error in said orientation information by offsetting said first position based on positions of predefined edges in said video.

12. A method according to claim 2, wherein:

said video is from a first camera of a set of multiple camera capturing a baseball game; and

said step of determining a first position includes receiving an indication of said first camera, accessing transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforming said location of said strike zone to a two dimensional position in said video.

13. A method according to claim 2, wherein:

said step of determining a first position includes correcting for error in said first position by offsetting said first position.

14. A method according to claim 2, wherein:

said step of determining a first position includes correcting for error in said first position by offsetting said first position based on positions of predefined edges in said video.

15. A method according to claim 2, wherein:

said video is from a first camera;

said step of determining a location of said strike zone is performed using a fourth camera; and

said step of determining a set of one or more locations of a ball is performed using a third camera.

16. A method according to claim 2, wherein:

said step of determining a location of said strike zone includes receiving positions of lines in an image of a batter, said lines corresponding to positions of said batter'"'"'s body.

17. A method according to claim 16, wherein:

said lines include a first line, a second line and a third line;

a first line is positioned at said batter'"'"'s feet;

a second line is positioned at said batter'"'"'s knee; and

a third line is positioned at said batter'"'"'s belt area.

18. A method according to claim 16, wherein:

said lines move in response to an input device such that a ratio of space between said lines remains constant.

19. A method according to claim 2, wherein:

said step of determining a location of said strike zone includes using one or more positions indicated on said batter in an image of said batter to determine height of said strike zone.

20. A method according to claim 19, wherein:

said video is from a first camera of a set of multiple camera capturing a baseball game; and

said step of determining a first position includes receiving an indication of said first camera, accessing transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforming said location of said strike zone to a two dimensional position in said video.

21. A method of providing strike zone information, comprising the steps of:

determining one or more locations of a ball; and

indicating within a video image whether one of said one or more locations of said ball intersects a strike zone;

wherein said step of determining includes the steps of;

receiving a set of video images that depict said ball, said video images include multiple data,

identifying which data of said multiple data corresponds to said ball using a series of filters, each filter potentially removes data from consideration as a candidate for said ball, said series of filters including a filter having a criteria that is independent of other images, and

using said data that corresponds to said ball to automatically determine said one or more locations.

22. A method according to claim 21, wherein:

said series of filters includes searching for data in a restricted area of said video images, searching for pixels in an appropriate color space, searching for minimum size clusters, searching for maximum size clusters, and searching for clusters having predefined shapes for a bounding box.

23. A method according to claim 21, wherein said step of identifying which data of said multiple data corresponds to said ball includes the steps of:

searching for clusters of data that are identifiable in subsequent fields of video; and

identifying matching clusters from two cameras.

24. A method according to claim 23, wherein said step of using said data includes the steps of:

using lines of position from matching clusters to identify three dimensional locations; and

filtering said three dimensional locations.

25. A method according to claim 21, further comprising the steps of:

determining a location of said strike zone;

determining a first position in a video, said first position corresponding to said location of said strike zone, wherein said step of determining the first position includes;

converting the three dimensional volume of said strike zone to two-dimensional locations within the video using data in video; and

adding an image for said strike zone to said video at said first position.

26. A method according to claim 21, further comprising the steps of:

determining a second position in said video, said second position corresponding to one of said set of one or more locations of said ball;

adding an image for said ball to said video based on said second position; and

determining whether said ball intersects said strike zone.

27. A method according to claim 25, wherein:

said video is from a first camera;

said first camera can change orientation;

said first camera includes one or more camera sensors for measuring orientation information of said first camera; and

said step of determining a first position uses said orientation information.

28. A method according to claim 27, wherein:

said step of determining a first position includes correcting for error in said orientation information be offsetting said first position based on positions of predefined edges in said video.

29. A method according to claim 25, wherein:

said video is from a first camera of a set of multiple camera capturing a baseball game; and

said step of determining a first position includes receiving an indication of said first camera, accessing transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforming said location of said strike zone to a two dimensional position in said video.

30. A method according to claim 25, wherein:

said step of determining a location of a strike zone includes using one or more positions indicated on said batter in an image of said batter to determine height of said strike zone.

31. A method of providing strike zone information, comprising the steps of:

determining a location of a strike zone for a first batter by receiving an indication of one or more positions on said first batter in an image of said first batter and using said indicated positions to automatically calculate height and a three dimensional volume of said strike zone;

determining a first position in a video, said first position corresponding to said location of said strike zone; and

adding an image for said strike zone to said video at said first position, wherein said step of determining the first position includes;

converting the three dimensional volume of said strike zone to two-dimensional locations within the video using data in video.

32. A method according to claim 31, wherein:

said video is from a first camera;

said first camera can change orientation;

said first camera includes one or more camera sensors for measuring orientation information of said first camera; and

said step of determining a first position uses said orientation information.

33. A method according to claim 32, wherein:

said step of determining a first position includes correcting for error in said orientation information be offsetting said first position based on positions of predefined edges in said video.

34. A method according to claim 31, wherein:

said video is from a first camera of a set of multiple camera capturing a baseball game; and

said step of determining a first position includes receiving an indication of said first camera, accessing transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforming said location of said strike zone to a two dimensional position in said video.

35. A method according to claim 31, wherein:

said step of receiving an indication includes receiving positions of lines in a video of a batter, said lines corresponding to positions of said batter'"'"'s body.

36. A method according to claim 35, wherein:

said lines include a first line, a second line and a third line;

a first line is positioned at said batter'"'"'s feet;

a second line is positioned at said batter'"'"'s knee; and

a third line is positioned at said batter'"'"'s belt area.

37. A method according to claim 35, wherein:

two or more of said lines move in response to an input device such a ratio of space between said lines remains constant.

38. An apparatus for providing strike zone information, comprising:

a set of image sensors arranged to sense data about a ball and a strike zone; and

one or more processors in communication with said image sensors, said one or more processors use data from said sensors to determine one or more locations of said ball and determine a location of said strike zone for a batter by using one or more positions indicated on said batter in an image of said batter to determine height of said strike zone, said one or more processors determine a first position in a video corresponding to said location of said strike zone and add a first image for said strike zone to said video at said first position, said one or more processors determine a second position in said video corresponding to one of said set of one or more locations of said ball and add a second image based on said second position, said one or more processors converting the height of said strike zone and one or more locations of said ball to two-dimensional locations within the video using data in the video.

39. An apparatus according to claim 38, wherein:

said one or more processors determine whether said ball intersects said strike zone and report whether said ball intersects said strike zone.

40. An apparatus according to claim 38, wherein:

said adding of a first image and adding of a second image includes adding graphics to said video to depict said strike zone and where said ball intersected said strike zone.

41. An apparatus according to claim 38, wherein:

said image sensors include a first camera used to determine said location of said strike zone and two additional cameras used to determined said one or more locations of said ball; and

said video is captured by a fourth camera.

42. An apparatus according to claim 38, wherein:

said video is from a first camera;

said first camera can change orientation;

said first camera includes one or more camera sensors for measuring orientation information of said first camera; and

said step of determining a first position uses said orientation information.

43. An apparatus according to claim 42, wherein:

said determining of said first position includes correcting for error in said orientation information be offsetting said first position based on positions of predefined edges in said video.

44. An apparatus according to claim 38, wherein:

said video is from a first camera of a set of multiple camera capturing a baseball game; and

said determining of said first position includes receiving an indication of said first camera, accessing transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforming said location of said strike zone to a two dimensional position in said video.

45. An apparatus according to claim 38, wherein:

said determining one or more locations of said ball includes the steps of;

receiving a set of video images that depict said ball, said video images include multiple data,

identifying which data of said multiple data corresponds to said ball using a series of filters, each filter potentially removes data from consideration as a candidate for said ball, and

using said data that to corresponds to said ball to automatically determine said one or more locations.

46. An apparatus for providing strike zone information, comprising:

a set of image sensors arranged to sense data about a ball and a strike zone;

a first set of one or more processors in communication with said image sensors, said first set of one or more processors use data from said sensors to determine one or more locations of said ball and determine a set of three dimensional coordinates representing a location of said strike zone;

a second processor in communication with said first set of one or more processors, said second processor receives an indication of a first camera of a set of cameras, accesses transformation data specific to said first camera from sets of transformation data specific to each of said cameras and transforms said one or more locations of said ball and said three dimensional coordinates representing said location of said strike zone to two dimensional positions in a video from said first camera;

a third processor in communication with said second processor, said third processor prepares an image of said strike zone and an image of said ball; and

a video modification unit in communication with said third processor and able to receive video from said cameras, said video modification unit receives said image of said strike zone and said image of said ball, said video modification unit adds said image of said strike zone and said image of said ball to said video from said first camera by converting the three dimensional coordinates of said strike zone and one or more locations of said ball to two-dimensional locations within said video using data in said video.

47. An apparatus according to claim 46, wherein:

said image sensors include a first camera used to determine said location of said strike zone and two additional cameras used to determined said one or more locations of said ball; and

said video is captured by a fourth camera.

48. An apparatus according to claim 46, wherein:

said determining of one or more locations of said ball includes the steps of;

receiving a set of video images that depict said ball, said video images include multiple data,

identifying which data of said multiple data corresponds to said ball using a series of filters, each filter potentially removes data from consideration as a candidate for said ball, and

using said data that to corresponds to said ball to determine said one or more locations.

49. An apparatus according to claim 46, wherein:

said determining of one or more locations of said ball includes the steps of;

receiving a set of video images that depict said ball,

searching for data in a restricted area of said video images, said restricted area being an area where said ball is expected to be during a pitch, and

using said data to determine said one or more locations.

50. An apparatus according to claim 46, wherein:

said video modification unit is a keyer.

51. An apparatus according to claim 46, further comprising:

a video delay device in communication with said video modification unit, said video delay device receives said video from said first camera, delays said video from said first camera and send said delayed video from said first camera to said video modification unit.