Tracking of handheld sporting implements using computer vision

• US 10,346,679 B2
• Filed: 10/19/2018
• Issued: 07/09/2019
• Est. Priority Date: 03/16/2016
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportsmedia Technology Corporation
• Inventors: Painter, James G.
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US16/165,432
• Publication Number:
  US 20190050636A1
• Time in Patent Office: 263 Days
• Field of Search: 382103
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A path and/or orientation of object approaching an athlete is tracked using two or more cameras. At least two sets of images of the object are obtained using at least two different cameras having different positions. Motion regions within images are identified, and candidate locations in 2D space of the object are identified within the motion region(s). Based thereon, a probable location in 3D space of the identifiable portion is identified, for each of a plurality of instants during which the object was approaching. A piecewise 3D trajectory of at least the identifiable portion of the object is approximated from the probable locations in 3D space of the object for multiple instants during which the object was approaching the athlete. A graphical representation of the 3D trajectory of the object is incorporated into at least one of the sets of images.

Claims

1. A method for tracking a sporting implement during a sporting event, comprising:

providing at least two cameras constructed and configured for network communication with at least one processor, wherein the at least two cameras are located at different positions in a sporting event facility;

the at least two cameras capturing at least two different sets of images of the sporting implement;

the at least one processor receiving the at least two different sets of images of the sporting implement from the at least two cameras;

the at least one processor identifying at least one motion region in the at least two different sets of images;

the at least one processor identifying a first location of at least one portion of the sporting implement within the at least one motion region;

the at least one processor identifying a second location of the at least one portion of the sporting implement based on the first location; and

the at least one processor identifying at least one combination of candidate locations of the at least one portion of the sporting implement, determining a corresponding line of position in 3D space for each candidate location in each combination of candidate locations, and determining a location from which distances to at least two of a multiplicity of lines of position are at a minimum as a probable location in 3D space of the at least one portion of the sporting implement;

wherein the first location comprises at least one candidate location, and wherein the second location comprises at least one probable location.

2. The method of claim 1, wherein the at least two different sets of images of the sporting implement comprise a full range of motion of the sporting implement.

3. The method of claim 1, further comprising the at least one processor approximating a trajectory of the sporting implement based on a multiplicity of probable locations of the at least one portion of the sporting implement identified for a multiplicity of instants.

4. The method of claim 3, further comprising the at least one processor superimposing a graphical representation of at least one part of the trajectory of the sporting implement over at least one of the at least two different sets of images;

and the at least one processor displaying the graphical representation of the at least one part of the trajectory of the sporting implement via a user interface display.

5. The method of claim 1, further comprising the at least one processor filtering out at least one false positive candidate location based on heuristics.

6. The method of claim 1, wherein the at least one portion of the sporting implement comprises a distal end of the sporting implement.

7. The method of claim 1, further comprising the at least one processor identifying an approximate center of elliptical arc motion of the sporting implement;

and the at least one processor identifying the at least one candidate location based on the approximate center of elliptical arc motion of the sporting implement.

8. The method of claim 1, further comprising the at least one processor identifying at least one motion region by comparing a sequence of images from the at least two different sets of images of the sporting implement.

9. The method of claim 1, wherein the sporting implement is a handheld sporting implement, and wherein the at least two different sets of images comprise a full stance of an athlete swinging the handheld sporting implement.

10. The method of claim 9, wherein the at least two different sets of images comprise an object being swung at moving toward the handheld sporting implement.

11. A system for tracking a sporting implement during a sporting event, comprising:

at least two cameras constructed and configured for network communication with at least one processor;

wherein the at least two cameras are located at different positions in a sporting event facility;

wherein the at least two cameras are configured to capture at least two different sets of images of the sporting implement;

wherein the at least one processor is configured to receive the at least two different sets of images of the sporting implement from the at least two cameras, identify at least one motion region in the at least two different sets of images, identify a first location of at least one portion of the sporting implement within the at least one motion region, and identify a second location of the at least one portion of the sporting implement based on the first location;

wherein the at least one processor is further configured to approximate a trajectory of the sporting implement based on a multiplicity of second locations of the at least one portion of the sporting implement identified for a multiplicity of instants; and

wherein the at least one processor is further configured to superimpose a graphical representation of the trajectory of the sporting implement over at least one of the at least two different sets of images, and wherein the at least one processor comprises a user interface display configured to display the graphical representation of the trajectory of the sporting implement.

12. The system of claim 11, wherein the sporting implement is a handheld sporting implement, wherein the handheld sporting implement comprises a distal end and a proximal end, and wherein the at least one portion of the sporting implement comprises the distal end of the handheld sporting implement.

13. A system for tracking a sporting implement during a sporting event, comprising:

at least two cameras constructed and configured for network communication with at least one processor;

wherein the at least two cameras are located at different positions in a sporting event facility;

wherein the at least two cameras are configured to capture at least two different sets of images of the sporting implement;

wherein the at least one processor is configured to receive the at least two different sets of images of the sporting implement from the at least two cameras, identify at least one motion region in the at least two different sets of images, identify at least one two-dimensional (2D) candidate location of at least one portion of the sporting implement within the at least one motion region, identify a three-dimensional (3D) probable location of the at least one portion of the sporting implement based on the at least one 2D candidate location, and approximate a 3D trajectory of the sporting implement based on a multiplicity of probable locations of the at least one portion of the sporting implement identified for a multiplicity of instants; and

wherein the at least one processor is further configured to superimpose a graphical representation of the 3D trajectory of the sporting implement over at least one of the at least two different sets of images.

14. The system of claim 13, wherein the 3D trajectory of the sporting implement comprises a path and an orientation of the sporting implement.

15. The system of claim 13, wherein the sporting implement is a handheld sporting implement, and wherein the at least one portion of the sporting implement comprises a distal end of the handheld sporting implement.