Jump shot and athletic activity analysis system

• US 10,664,690 B2
• Filed: 03/06/2017
• Issued: 05/26/2020
• Est. Priority Date: 11/21/2013
• Status: Active Grant

• Current Assignee: Mo' Motion Ventures
• Original Assignee: Mo' Motion Ventures
• Inventors: Holohan, Maureen
• Primary Examiner(s): Park, Soo Jin
• Application Number:
  US15/451,044
• Publication Number:
  US 20170177930A1
• Time in Patent Office: 1,177 Days
• Field of Search: None
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2225/20 with means for remote communication, e.g. internet or the like
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 69/0071 for basketball
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

Techniques where a computer or mobile device performs video analysis of a person performing a physical activity such as a basketball jump shot are described. The computer or mobile device performs video analysis based on one or more reference skeletons. The reference skeleton may be superimposed over captured video of the person performing the physical activity or compared to a generated player skeleton based on the captured video to determine derivation from the reference skeleton in the physical activity performed by the person.

Claims

1. A computer program product for video analysis of a basketball jump shot, the computer program product comprising:

a plurality of processor-executable instructions configured for processing video data, the video data comprising a plurality of frames of a person performing the basketball jump shot, the instructions being resident on a non-transitory computer-readable storage medium and being configured, upon execution by a processor, to;

determine a frame of the video data that depicts a body position of the person in a phase of the basketball jump shot, wherein the phase comprises a member of the group consisting of (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase;

process pixels in the determined frame to determine a body position for the person within the determined frame, wherein the determined body position comprises data indicative of an arrangement of joints and limbs for the person within the determined frame;

compare the determined body position in the determined frame with a reference skeleton for the basketball jump shot phase;

determine a deviation between the determined body position and the reference skeleton based on the comparison operation;

repeat the frame determination operation, the processing operation, the comparison operation, and the deviation determination operation for a plurality of basketball jump shots by the person;

maintain data indicative of whether the basketball jump shots went through a basketball hoop;

build a database associated with the basketball jump shots, the database including the determined deviations for the basketball jump shots in association with data indicative of which of the basketball jump shots went through the basketball hoop;

determine a plurality of different types of deviations with respect to a plurality of the basketball jump shots based on the repeated operations; and

analyze the database to correlate the determined deviation types for the basketball jump shots with respect to the associated data indicative of whether the basketball jump shots went through the basketball hoop and assess which deviation types are correlated with at least one of made shots and missed shots.

2. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

analyze the database to generate metrics about the person'"'"'s basketball jump shots; and

communicate the generated metrics for presentation to a user.

3. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

identify which determined deviation types exhibit a correlation beyond a threshold with respect to basketball jump shots that did not go through the basketball hoop.

4. The computer program product of claim 1 wherein the reference skeleton is representative of a desired body position with respect to a fundamental aspect of the basketball jump shot phase, and wherein the deviation is representative of a difference in the fundamental aspect as between the determined body position and the desired body position of the reference skeleton.

5. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

perform the frame determination operation, the processing operation, the comparison operation, and the deviation determination operation with respect to a plurality of the basketball jump shot phases such that the frame determination operation includes (1) determining a first frame that depicts the body position of the person for a member of the group consisting of the base phase, the frame phase, the release phase, the follow-through phase, and the finish phase, and (2) determining a second frame that depicts the body position of the person for another member of the group consisting of the base phase, the frame phase, the release phase, the follow-through phase, and the finish phase.

6. The computer program product of claim 5 wherein the phases comprise (1) the base phase, (2) the frame phase, (3) the release phase, (4) the follow-through phase, and (5) the finish phase, and wherein the plurality of processor-executable instructions are further configured to:

determine the first frame as a frame that depicts the body position of the person for the base phase;

determine the second frame as a frame that depicts the body position of the person for the frame phase;

determine a third frame that depicts the body position of the person for the release phase;

determine a fourth frame as a frame that depicts the body position of the person for the follow-through phase; and

determine a fifth frame that depicts the body position of the person for the finish phase.

7. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

communicate a result based on the determined deviation.

8. The computer program product of claim 1 wherein the plurality of processor-executable instructions are further configured to:

superimpose the reference skeleton over the determined body position.

9. The computer program product of claim 1 wherein the video data exhibits a perspective with respect to the person, and wherein the perspective comprises at least one of a front view, a side view, a back view, and an angle view.

10. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

perform the frame determination operation, the processing operation, the comparison operation, and the deviation determination operation with respect to first video data from a first perspective and second video data from a second perspective to determine the deviation based on a 3D model of the basketball jump shot.

11. The computer program product of claim 1 wherein the determined frame comprises a plurality of frames that depict the person'"'"'s body position in the basketball jump shot phase.

12. The computer program product of claim 1, wherein the plurality of processor-executable instructions are further configured to:

calculate body edge coordinates and a body size for the person in the basketball jump shot phase based on the processed pixels of the determined frame;

determine joint coordinates for the person in the basketball jump shot phase based on the body edge coordinates and the body size; and

perform the comparison based on the determined joint coordinates relative to the reference skeleton.

13. The computer program product of claim 1 wherein the determined deviation types include a falling backward deviation.

14. The computer program product of claim 1 wherein the determined deviation types include an elbow flare out deviation.

15. The computer program product of claim 1 wherein the determined deviation types include a falling or leaning forward deviation.

16. The computer program product of claim 1 wherein the determined deviation types include a release point too low deviation.

17. The computer program product of claim 1 wherein the determined deviation types include a not square to the basketball hoop deviation.

18. A method for video analysis of a basketball jump shot, the method comprising:

processing video data, the video data comprising a plurality of frames of a person performing the basketball jump shot, wherein the processing step comprises;

a processor determining a frame of the video data that depicts a body position of the person in a phase of the basketball jump shot, wherein the phase comprises a member of the group consisting of (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase;

the processor processing pixels in the determined frame to determine a body position for the person within the determined frame, wherein the determined body position comprises data indicative of an arrangement of joints and limbs for the person within the determined frame;

the processor comparing the determined body position in the determined frame with a reference skeleton for the phase of the basketball jump shot phase; and

the processor determining a deviation between the determined body position and the reference skeleton based on the comparing step;

the processor repeating the frame determining step, the pixel processing step, the comparing step, and the deviation determining step for a plurality of basketball jump shots by the person;

the processor maintaining data indicative of whether the basketball jump shots went through a basketball hoop;

the processor building a database associated with the basketball jump shots, the database including the determined deviations for the basketball jump shots in association with data indicative of which of the basketball jump shots went through the basketball hoop;

the processor determining a plurality of different types of deviations with respect to a plurality of the basketball jump shots based on the repeated operations; and

the processor analyzing the database to correlate the determined deviation types for the basketball jump shots with respect to the associated data indicative of whether the basketball jump shots went through the basketball hoop and assess which deviation types are correlated with at least one of made shots and missed shots.

19. The method of claim 18 further comprising:

the processor identifying which determined deviation types exhibit a correlation beyond a threshold with respect to basketball jump shots that did not go through the basketball hoop.

20. The method of claim 18 wherein the determined deviation types include a falling backward deviation.

21. The method of claim 18 wherein the determined deviation types include an elbow flare out deviation.

22. The method of claim 18 wherein the determined deviation types include a falling or leaning forward deviation.

23. The method of claim 18 wherein the determined deviation types include a release point too low deviation.

24. A system for video analysis of a basketball jump shot, the system comprising:

a processor configured for processing video data, the video data comprising a plurality of frames of a person performing the basketball jump shot, wherein the processor is configured to;

determine a frame of the video data that depicts a body position of the person in a phase of the basketball jump shot, wherein the phase comprises a member of the group consisting of (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase;

process pixels in the determined frame to determine a body position for the person within the determined frame, wherein the determined body position comprises data indicative of an arrangement of joints and limbs for the person within the determined frame;

compare the determined body position in the frame with a reference skeleton for the basketball jump shot phase;

determine a deviation between the determined body position and the reference skeleton based on the comparison operation;

repeat the frame determination operation, the processing operation, the comparison operation, and the deviation determination operation for a plurality of basketball jump shots by the person;

maintain data indicative of whether the basketball jump shots went through a basketball hoop;

build a database associated with the basketball jump shots, the database including the determined deviations for the basketball jump shots in association with data indicative of which of the basketball jump shots went through the basketball hoop;

determine a plurality of different types of deviations with respect to a plurality of the basketball jump shots based on the repeated operations; and

analyze the database to correlate the determined deviation types for the basketball jump shots with respect to the associated data indicative of whether the basketball jump shots went through the basketball hoop and assess which deviation types are correlated with at least one of made shots and missed shots.

25. The system of claim 24 further comprising:

a camera for cooperation with the processor, the camera configured to generate the video data.

26. The system of claim 25 wherein the camera comprises (1) a first camera configured to generate video data corresponding to a front view of the person performing the basketball jump shot, and (2) a second camera configured to generate video data corresponding to a side view of the person performing the basketball jump shot, and wherein the determined frame includes a front view frame corresponding to the phase and a side view frame corresponding to the phase.

27. The system of claim 26 wherein the camera comprises (1) a baseline camera, (2) a first sideline camera, and (3) a second sideline camera, wherein the first and second sideline cameras are positioned on opposite sidelines of the same side of a basketball court as the baseline camera.

28. The system of claim 24 wherein the processor further configured to identify which determined deviation types exhibit a correlation beyond a threshold with respect to basketball jump shots that did not go through the basketball hoop.

29. The system of claim 24 wherein the determined deviation types include a falling backward deviation.

30. The system of claim 24 wherein the determined deviation types include an elbow flare out deviation.

31. The system of claim 24 wherein the determined deviation types include a falling or leaning forward deviation.

32. The system of claim 24 wherein the determined deviation types include a release point too low deviation.

33. The system of claim 24 wherein the determined deviation types include a not square to the basketball hoop deviation.