Tracking of handheld sporting implements using computer vision

• US 10,115,007 B2
• Filed: 12/18/2017
• Issued: 10/30/2018
• Est. Priority Date: 03/16/2016
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportsmedia Technology Corporation
• Inventors: Painter, James G.
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/845,523
• Publication Number:
  US 20180107868A1
• Time in Patent Office: 316 Days
• Field of Search: 382103, 382107, 482 1, 482901, 702141
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

A path and/or orientation of object approaching an athlete is tracked using two or more cameras. At least two sets of video images of the object are obtained using at least two different cameras having different positions. Motion regions within video images are identified, and candidate locations in 2D space of the object is/are identified within the motion region(s). Based thereon, a probable location in 3D space of the identifiable portion is identified, for each of a plurality of instants during which the object was approaching. A piecewise 3D trajectory of at least the identifiable portion of the object is approximated from the probable locations in 3D space of the object for multiple instants during which the object was approaching the athlete. A graphical representation of the 3D trajectory of the object is incorporated into at least one of the sets of video images.

Claims

1. A method for tracking an object approaching an athlete, the method comprising:

at least three different cameras having different positions capturing three or more different sets of video images of the object approaching the athlete, wherein the athlete is in a stance prepared to strike the object;

a server processor receiving the three or more different sets of video images, wherein the three or more different sets of video images comprise the stance of the athlete;

the server processor approximating at least part of a 3D trajectory of the object based on the three or more sets of video images; and

the server processor superimposing a graphical representation of the at least part of the 3D trajectory of the object over at least one of the three or more different sets of video images.

2. The method of claim 1, wherein the object is a ball.

3. The method of claim 1, wherein the object is a hockey puck.

4. The method of claim 1, further comprising:

identifying one or more motion regions within the three or more different sets of video images;

identifying one or more candidate locations in a two-dimensional (2D) space of an identifiable portion of the athlete and the object within the one or more motion regions, for at least a subset of the three or more different sets of video images; and

identifying, based on the one or more candidate locations in a 2D space of the athlete and the object, a probable location in three-dimensional (3D) space of the athlete and the object, for each of a plurality of instants during which the object is approaching the athlete.

5. The method of claim 1, further comprising approximating an outline of the athlete.

6. The method of claim 1, further comprising the at least three different cameras capturing the athlete swinging at the object with a handheld sporting implement.

7. The method of claim 6, wherein the three or more different sets of video images include a full range of the swinging motion.

8. The method of claim 6, further comprising:

approximating at least part of a 3D trajectory of the handheld sporting implement based on the three or more different sets of video images; and

incorporating a graphical representation of the 3D trajectory of the handheld sporting implement in at least one of the three or more different sets of video images comprising the stance of the athlete.

9. A system for tracking an object approaching an athlete, the system comprising:

a first camera and a second camera constructed and configured in network communication with at least one processor;

wherein the first camera and second camera are operable to capture video images of the athlete and the object approaching the athlete, the athlete in a stance prepared to strike the object, wherein the first and second cameras have different positions;

wherein the at least one processor is operable to;

receive two or more different sets of video images from the first camera and the second camera;

identify one or more motion regions within each of the two or more different sets of video images;

identify one or more candidate locations in two-dimensional (2D) space of an identifiable portion of the object within the one or more motion regions of the video images, for at least a subset of the two or more different sets of video images;

identify, based on the one or more candidate locations in 2D space of the identifiable portion of the object, a probable location in three-dimensional (3D) space of the identifiable portion of the object, for each of a plurality of instants during which the object is approaching the athlete; and

approximate a piecewise 3D trajectory of at least the identifiable portion of the object from the probable locations in 3D space of the identifiable portion of the object identified for the plurality of instants during which the object is approaching the athlete; and

incorporate a graphical representation of the 3D trajectory of the object into at least one of the two or more different sets of video images.

10. The system of claim 9, further including a third camera operable to capture a set of video images of the athlete and the object approaching the athlete and transmit the set of video images to the at least one processor.

11. The system of claim 9, wherein the at least one processor is operable to identify the one or more motion regions by comparing a video image to at least one of a preceding or following video image.

12. The system of claim 9, wherein the at least one processor is operable to approximate the piecewise 3D trajectory based on a curve fitting algorithm.

13. The system of claim 12, wherein the at least one processor is operable to determine the two or more lines of position based on transformation matrices.

14. The system of claim 9, wherein the at least one processor identifies a probable location in 3D space of the object, for each of a plurality of instants during which the object is approaching the athlete, by:

identifying different combinations of candidate locations in 2D space of the probable location of the object that were based on images captured using at least two different ones of the cameras for an instant;

determining two or more lines of position for each of at least two of the combinations; and

determining, as the probable location in 3D space of object for the instant, a location in 3D space where distances between the location and the two or more lines of position are at their minima.