Stereoscopic image capture with performance outcome prediction in sporting environments

• US 9,390,501 B2
• Filed: 05/22/2015
• Issued: 07/12/2016
• Est. Priority Date: 05/24/2007
• Status: Active Grant

• Current Assignee: Pillar Vision Incorporated
• Original Assignee: Pillar Vision Incorporated
• Inventors: Marty, Alan W., Edwards, Thomas A., Carter, John
• Primary Examiner(s): LIDDLE, JAY TRENT
• Application Number:
  US14/720,389
• Publication Number:
  US 20150332450A1
• Time in Patent Office: 417 Days
• US Class Current: 1/1
• CPC Class Codes: A63B 2024/0034 during flight
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 69/00 Training appliances or apparatus for special sports training of parachutists B64D23/00
• CPC Class Codes: A63B 69/0071 for basketball
• CPC Class Codes: A63B 69/38 for tennis A63B61/006, A63B69/0073 and A63B69/0097 take precedence
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Methods and apparatus relating to predicting outcome in a sporting environment are described. The methods and apparatus are used to relate trajectory performance of an object to body motions and body orientation associated with a generating the trajectory of the object. When equipment is utilized to generate the trajectory of an object, than the effects of equipment motions and equipment orientation can be also related to trajectory performance. The method and apparatus can be used to predict body motions and body orientations that increase the likelihood of achieving a desired outcome including specifying optimum motions and orientations for a particular individual. The method and apparatus may be used in training, coaching and broadcasting environments.

Claims

1. A method for basketball, comprising:

capturing, with a camera, a plurality of images of a body of a first user while the first user is performing or simulating a basketball shot;

identifying, by a processor, at least one body part of the first user in the captured images;

determining, by the processor, a parameter indicative of motion of the at least one body part of the first user during the basketball shot based on the images;

storing, in memory prior to the capturing, data that is based on motions of at least one body part of at least one user during a plurality of basketball shots by the at least one user;

comparing, by the processor, the parameter to the data;

determining, by the processor, a deviation between the parameter and the data based on the comparing, the deviation indicative of a difference in the motion of the at least one body part of the first user during the basketball shot relative to the motions of the at least one body part of the at least one user during the plurality of basketball shots;

predicting, by the processor, a trajectory outcome of a physical basketball or a virtual basketball based on the deviation, wherein the trajectory outcome includes at least one of the group including (1) whether the physical basketball passes through a physical hoop or the virtual basketball passes through a virtual hoop and (2) whether the physical basketball contacts the physical hoop or the virtual basketball contacts the virtual hoop; and

providing an output indicative of the predicted trajectory outcome.

2. The method of claim 1, wherein the camera comprises an RGB-Z camera.

3. The method of claim 2, further comprising determining with the RGB-Z camera a distance of the first user from the RGB-Z camera.

4. The method of claim 2, wherein the RGB-Z camera is mounted on a backboard coupled to the hoop or on a pole coupled to the backboard.

5. The method of claim 1, further comprising determining, by the processor, a position of the at least one body part of the first user relative to a position of the physical hoop or the virtual hoop, wherein the predicting is based on the determined position of the at least one body part of the first user relative to the position of the physical hoop or the virtual hoop.

6. The method of claim 5, further comprising determining, by the processor, a range for the parameter based on the data and the relative position of the at least one body part of the first user to the position of the physical hoop or the virtual hoop, wherein the comparing comprises determining, by the processor, whether the parameter is within the range.

7. The method of claim 6, wherein the determining the position of the at least one body part of the first user relative to the position of the physical hoop is based on at least one image captured with the camera.

8. The method of claim 1, wherein the method further comprises determining, by the processor, forces applied by the body to the physical basketball based on the motion of the at least one body part of the first user, and wherein the predicting is based on the forces.

9. The method of claim 1, wherein the at least one body part of the first user includes a hand of the first user.

10. The method of claim 1, further comprising determining, by the processor, an orientation of the body of the first user during the basketball shot based on the images, wherein the predicting is based on the determined orientation.

11. The method of claim 1, wherein the predicting comprises predicting a trajectory of the physical basketball or the virtual basketball based on the parameter, wherein the method further comprises comparing the trajectory to a position of the physical hoop or the virtual hoop, and wherein the predicting is based on the comparing.

12. The method of claim 1, further comprising:

predicting, by the processor, a trajectory of the physical basketball or the virtual basketball based on the captured images; and

comparing, by the processor, the predicted trajectory to a location of the physical hoop or the virtual hoop,

wherein the predicting the trajectory outcome is based on the comparing the trajectory to the location of the physical hoop or the virtual hoop.

13. The method of claim 12, further comprising predicting, based on the comparing the predicted trajectory to the location of the physical hoop or the virtual hoop, a point at which the physical basketball enters or contacts the physical hoop or the virtual basketball enters or contacts the virtual hoop, wherein the predicting the trajectory outcome is based on the predicted point.

14. A method for basketball, comprising:

capturing, with a camera, a plurality of images of a body of a first user while the first user is performing a basketball shot;

identifying, by a processor, at least one body part of the first user in the captured images;

determining, by the processor, a parameter indicative of motion of the at least one body part of the first user during the basketball shot based on the images;

storing, in memory prior to the capturing, data that is based on motions of at least one body part of at least one user during a plurality of basketball shots by the at least one user;

comparing, by the processor, the parameter to the data;

determining, by the processor, a deviation between the parameter and the data based on the comparing, the deviation indicative of a difference in the motion of the at least one body part of the first user during the basketball shot relative to the motions of the at least one body part of the at least one user during the plurality of basketball shots;

predicting, by the processor, a trajectory outcome of a physical basketball based on the deviation, wherein the trajectory outcome includes at least one of the group including whether the physical basketball passes through a physical hoop and whether the physical basketball contacts the physical hoop; and

providing an output indicative of the predicted trajectory outcome.

15. A system, comprising:

a camera configured to capture a plurality of images of a body of a first user while the first user is performing or simulating a basketball shot;

memory for storing data that is based on motions of at least one body part of at least one user during a plurality of basketball shots by the at least one user prior to the basketball shot performed or simulated by the first user;

a processor programmed to identify at least one body part of the first user in the captured images and to determine, based on the images, a parameter indicative of a motion of the at least one body part of the first user during the basketball shot performed or simulated by the first user, the processor further programmed to perform a first comparison between the parameter and the data and to determine a deviation between the parameter and the data, the deviation indicative of a difference in the motion of the at least one body part of the first user during the basketball shot relative to the motions of the at least one body part of the at least one user during the plurality of basketball shots, the processor further programmed to predict a trajectory outcome of a physical basketball or a virtual basketball based on the first comparison, wherein the trajectory outcome includes at least one of the group including (1) whether the physical basketball passes through a physical hoop or the virtual basketball passes through a virtual hoop and (2) whether the physical basketball contacts the physical hoop or the virtual basketball contacts the virtual hoop; and

an interface configured to provide an output indicative of the trajectory outcome.

16. The system of claim 15, wherein the interface comprises a sound projection device.

17. The system of claim 15, wherein the interface comprises a display device.

18. The system of claim 15, wherein the camera comprises an RGB-Z camera.

19. The system of claim 18, wherein the RGB-Z camera is configured to determine a distance of the first user from the RGB-Z camera.

20. The system of claim 19, wherein the processor is programmed to determine, based on the distance, a position of the at least one body part of the first user relative to a position of the physical hoop or the virtual hoop.

21. The system of claim 15, wherein the processor is further programmed to determine, based on the images, an orientation of the body of the first user during the basketball shot performed or simulated by the first user and to predict the trajectory outcome based on the determined orientation.

22. The system of claim 15, wherein the processor is further programmed to perform a second comparison between the trajectory and a position of the physical hoop or the virtual hoop, and wherein the processor is programmed to predict the trajectory outcome based on the second comparison.