Video tracking of baseball players which identifies merged participants based on participant roles

• US 9,007,463 B2
• Filed: 12/22/2010
• Issued: 04/14/2015
• Est. Priority Date: 12/22/2010
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportsvision, Inc.
• Inventors: Elangovan, Vidya, Fecho, Marlena, Heidmann, Timothy P.
• Primary Examiner(s): Holder, Anner
• Assistant Examiner(s): Barazesh, Ellyar Y
• Application Number:
  US12/976,758
• Publication Number:
  US 20120162434A1
• Time in Patent Office: 1,574 Days
• Field of Search: None
• US Class Current: 348/157
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds

Abstract

Video frames of a baseball game are analyzed to determine a track for the participants in the game and to update a digital record of the game. The merging of participants in a video frame is resolved by associating the participants'"'"' tracks before and/or after the merging with a most likely participant role, such as a player, coach or umpire role. The role of one merged participant can be used to deduce the role of the other merged participant. In this way, the digital record can be completed even for the merged period. The role of a participant can be based, e.g., on the location of the participant relative to a base, a coach'"'"'s box region, a pitcher'"'"'s mound, a dugout, or a fielding position, or by determining that a participant is running along a path to a base or performing some other movement.

Claims

1. Tangible, non-transitory computer readable storage having computer readable software embodied thereon for programming at least one processor to perform a method for tracking participants in a baseball game at a baseball park, the baseball park includes a playing field, the method performed comprises:

defining first and second areas around a landmark on the playing field, the first and second areas each have a specified size and are positioned in specified way with respect to the landmark, and the second area is larger than the first area;

defining a region of the baseball park, the region is outside the first and second areas;

analyzing video frames obtained from one or more cameras, the video frames provide images of the participants in the baseball park;

based on the analyzing;

determining a merged track of at least first and second participants in the baseball park, the merged track indicates successive locations of the at least first and second participants in the baseball park during a merged time interval in which the first and second participants are merged in the video frames, determining one non-merged track of the first participant in the baseball park during one non-merged time interval adjacent to the merged time interval in which the first and second participants are not merged in the video frames, and determining one non-merged track of the second participant in the baseball park during the one non-merged time interval;

determining that the first participant remains within the first area during the one non-merged time interval based on the one non-merged track of the first participant;

determining that the second participant performs a movement during the one non-merged time based on the one non-merged track of the second participant, the movement is at least one of;

from the region to the second area or from the second area to the region;

identifying a role of the first participant from among a plurality of roles in the baseball game, the identifying the role of the first participant comprises determining that the first participant is associated with the landmark, the determining that the first participant is associated with the landmark is based on the determining that the first participant remains within the first area during the one non-merged time interval;

identifying a role of the second participant from among the plurality of roles, the identifying the role of the second participant comprises determining that the second participant is associated with the region, the determining that the second participant is associated with the region is based on the determining that the second participant performs the movement during the one non-merged time interval; and

responsive to the identifying the role of the first participant and the identifying the role of the second participant, updating a digital record of the baseball game to associate the first participant with the merged track and the one non-merged track of the first participant, and to indicate that the first participant was merged with the second participant in the merged track.

2. The tangible, non-transitory computer readable storage of claim 1, wherein the method performed further comprises:

responsive to the identifying the role of the first participant and the identifying the role of the second participant, updating the digital record to associate the second participant with the merged track and with the one non-merged track of the second participant and to indicate that the second participant was merged with the first participant in the merged track.

3. The tangible, non-transitory computer readable storage of claim 1, wherein:

the identifying of the role of the second participant is based on the identifying the role of the first participant.

4. The tangible, non-transitory computer readable storage of claim 1, wherein:

the landmark comprises a base;

the role of the first participant is a baseman of the base; and

the role of the second participant is a base runner of the base.

5. The tangible, non-transitory computer readable storage of claim 1, wherein:

the landmark comprises a base;

the role of the first participant is a base runner of the base; and

the role of the second participant is a coach.

6. The tangible, non-transitory computer readable storage of claim 1, wherein:

the landmark comprises a pitcher'"'"'s mound;

the role of the first participant is a pitcher;

the role of the second participant is a coach;

the first area is around the pitcher'"'"'s mound;

the second area is around the first area; and

the region comprises a dugout.

7. The tangible, non-transitory computer readable storage of claim 1, wherein:

the landmark comprises at least one of an on-deck circle, a coach'"'"'s box, a catcher'"'"'s box or a batter'"'"'s box.

17. The tangible, non-transitory computer readable storage of claim 1, wherein the method performed further comprises:

responsive to the identifying the role of the first participant and the identifying the role of the second participant, updating the digital record to indicate a time interval of the one non-merged time interval of the second participant used to resolve the merged track.

18. The tangible, non-transitory computer readable storage of claim 1, wherein the method performed further comprises:

based on the analyzing;

determining another non-merged track of the first participant in the baseball park during another non-merged time interval adjacent to the merged time interval in which the first and second participants are not merged, and determining another non-merged track of the second participant in the baseball park during the another non-merged time interval;

determining that the first participant remains within the first area during the another non-merged time interval based on the another non-merged track of the first participant;

determining that the second participant performs a movement from the region to the second area based on the one non-merged track of the second participant; and

determining that the second participant performs a movement from the second area to the region based on the another non-merged track of the second participant;

wherein;

the determining that the first participant is associated with the landmark is based on the determining that the first participant remains within the first area during the another non-merged time interval; and

the determining that the second participant is associated with the region is based on the determining that the second participant performs the movement from the region to the second area and the determining that the second participant performs the movement from the second area to the region.

8. A method for tracking participants in a baseball game at a baseball park, the baseball park includes a playing field, the method comprising the processor-implemented steps of:

analyzing a plurality of successive video frames obtained from one or more cameras, the video frames provide images of the participants in the baseball park;

based on the analyzing;

determining, among the plurality of successive video frames, video frames comprising a representation of a first participant merged with a representation of a second participant in the baseball park during a merged time interval, video frames comprising a representation of the first participant not merged with another participant in a non-merged time interval, and video frames comprising a representation of the second participant not merged with another participant in the non-merged time interval;

determining a merged track in the baseball park of the first and second participants based on the video frames comprising the representation of the first participant merged with the representation of the second participant;

determining a non-merged track in the baseball park of the first participant based on the video frames comprising the representation of the first participant not merged with another participant;

determining that the non-merged track of the first participant remains within a first area around a landmark on the playing field;

determining a non-merged track in the baseball park of the second participant based on the video frames comprising the representation of the second participant not merged with another participant;

determining that the non-merged track of the second participant indicates a movement of the second participant, the movement is at least one of;

from a second area around the landmark to a region of the baseball park outside the second area, the second area is larger than the first area, or from the region to the second area;

based on the determining that the non-merged track of the first participant remains within the first area, identifying a role of the first participant in the baseball game as being associated with the landmark and updating a digital record associated with the role of the first participant with locations of the non-merged track of the first participant cross-referenced to the non-merged time interval and to an indication that the first participant is not merged with another participant during the non-merged time interval; and

based on the determining that the non-merged track of the second participant indicates the movement of the second participant, identifying a role of the second participant in the baseball game as being associated with the region and updating a digital record associated with the role of the second participant with locations of the non-merged track of the second participant cross-referenced to the non-merged time interval and to an indication that the second participant is not merged with another participant during the non-merged time interval.

9. The method of claim 8, further comprising:

updating the digital record associated with the role of the second participant with the locations of the merged track cross-referenced to;

the merged time interval, an indication that a merger involving the second participant has been resolved and an indication that the merger involving the second participant was with the first participant.

10. The method of claim 8, wherein:

the identifying the role of the second participant is based on the role of the first participant; and

the identifying the role of the first participant is based on the role of the second participant.

11. The method of claim 8, further comprising:

updating the digital record associated with the role of the first participant to identify a time interval of the non-merged time interval which was used to identify the role of the first participant.

12. The method of claim 8, wherein:

the landmark comprises a base;

the region comprises a coach'"'"'s box region associated with the base;

the role of the first participant is a baseman of the base; and

the role of the second participant is a base coach of the base.

13. The method of claim 8, further comprising:

updating the digital record associated with the role of the first participant to identify a time interval of the non-merged time interval which was used to resolve the merged track.

14. The method of claim 8, wherein:

the landmark comprises a base;

the role of the first participant is a baseman of the base; and

the role of the second participant is a base runner.

15. The method of claim 8, wherein:

the landmark comprises a pitcher'"'"'s mound;

the region comprises a dugout;

the role of the first participant is a pitcher; and

the role of the second participant is a coach.

16. The method of claim 8, wherein:

the landmark comprises a base;

the role of the first participant is a baseman of the base;

the region comprises a dugout; and

the role of the second participant is a base runner.

19. An apparatus for tracking participants in a game at a playing field, the apparatus comprising:

a processor readable storage device having processor readable code embodied thereon; and

a processor, the processor is configured to execute the processor readable code to;

analyze a plurality of successive video frames obtained from one or more cameras, the video frames provide images of the participants in the game;

based on the analyzing;

determine, among the plurality of successive video frames, video frames comprising a representation of a first participant merged with a representation of a second participant in the game during a merged time interval, video frames comprising a representation of the first participant not merged with another participant in a non-merged time interval, and video frames comprising a representation of the second participant not merged with another participant in the non-merged time interval;

determine a merged track in the game of the first and second participants based on the video frames comprising the representation of the first participant merged with the representation of the second participant;

determine a non-merged track in the game of the first participant based on the video frames comprising the representation of the first participant not merged with another participant;

determine that the non-merged track of the first participant remains within a first area around a landmark on the playing field;

determine a non-merged track in the game of the second participant based on the video frames comprising the representation of the second participant not merged with another participant;

determine that the non-merged track of the second participant indicates a movement of the second participant, the movement is at least one of;

from a second area around the landmark to a region of the game outside the second area, the second area is larger than the first area, or from the region to the second area;

based on the determining that the non-merged track of the first participant remains within the first area, identify a role of the first participant in the game as being associated with the landmark and updating a digital record associated with the role of the first participant with locations of the non-merged track of the first participant cross-referenced to the non-merged time interval and to an indication that the first participant is not merged with another participant during the non-merged time interval; and

based on the determining that the non-merged track of the second participant indicates the movement of the second participant, identify a role of the second participant in the game as being associated with the region and updating a digital record associated with the role of the second participant with locations of the non-merged track of the second participant cross-referenced to the non-merged time interval and to an indication that the second participant is not merged with another participant during the non-merged time interval.

20. The apparatus of claim 19, wherein the processor is configured to execute the processor readable code to:

update the digital record associated with the role of the second participant with the locations of the merged track cross-referenced to;

the merged time interval, an indication that a merger involving the second participant has been resolved and an indication that the merger involving the second participant was with the first participant.

21. The apparatus of claim 19, wherein:

the identifying the role of the second participant is based on the role of the first participant; and

the identifying the role of the first participant is based on the role of the second participant.

22. The apparatus of claim 19, wherein the processor is configured to execute the processor readable code to:

update the digital record associated with the role of the first participant to identify a time interval of the non-merged time interval which was used to identify the role of the first participant.

23. The apparatus of claim 19, wherein the processor is configured to execute the processor readable code to:

update the digital record associated with the role of the first participant to identify a time interval of the non-merged time interval which was used to resolve the merged track.

24. The apparatus of claim 19, wherein:

the game is a baseball game.

25. The apparatus of claim 24, wherein:

the landmark comprises a base;

the region comprises a coach'"'"'s box region associated with the base;

the role of the first participant is a baseman of the base; and

the role of the second participant is a base coach of the base.

26. The apparatus of claim 24, wherein:

the landmark comprises a base;

the role of the first participant is a baseman of the base; and

the role of the second participant is a base runner.

27. The apparatus of claim 24, wherein:

the landmark comprises a pitcher'"'"'s mound;

the region comprises a dugout;

the role of the first participant is a pitcher; and

the role of the second participant is a coach.

28. The apparatus of claim 24, wherein:

the landmark comprises a base;

the role of the first participant is a baseman of the base;

the region comprises a dugout; and

the role of the second participant is a base runner.