Jump Shot and Athletic Activity Analysis System

• US 20150139502A1
• Filed: 11/21/2013
• Published: 05/21/2015
• Est. Priority Date: 11/21/2013
• Status: Active Grant

• Current Assignee: Mo' Motion Ventures
• Original Assignee: Mo' Motion Ventures
• Inventors: Holohan, Maureen
• Granted Patent: US 9,589,207 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G16H 20/30 relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264

Abstract

Techniques where a computer or mobile device performs video analysis of a person performing a physical activity are described. The computer or mobile device performs video analysis based on one or more reference skeletons. The reference skeleton may be superimposed over captured video of the person performing the physical activity or compared to a generated player skeleton based on the captured video to determine derivation from the reference skeleton in the physical activity performed by the person.

Claims

1. A computer program product for video analysis of a physical activity, the computer program product comprising:

a plurality of processor-executable instructions configured for processing video data, the video data comprising a plurality of frames of a person performing the physical activity, the instructions being resident on a non-transitory computer-readable storage medium and being configured, upon execution by a processor, to;

determine a frame of the video data that depicts a body position of the person in a phase of the physical activity;

compare the body position in the frame with a reference skeleton for the phase of the physical activity; and

determine a deviation between the frame body position and the reference skeleton based on the comparison operation.

2. The computer program product of claim 1 wherein the instructions are configured for execution by a mobile computing device.

3. The computer program product of claim 1 wherein the instructions are configured for execution by a server system.

4. The computer program product of claim 1 wherein the reference skeleton is representative of a desired body position with respect to a fundamental aspect of the physical activity phase, and wherein the deviation is representative of a difference in the fundamental aspect as between the frame body position and the desired body position of the reference skeleton.

5. The computer program product of claim 4 wherein the physical activity comprises a basketball jump shot.

6. The computer program product of claim 5 wherein the phase comprises a member of the group consisting of (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase.

7. The computer program product of claim 5, wherein the plurality of processor-executable instructions being further configured to:

perform the frame determining step, the comparing step, and the deviation determining step with respect to a plurality of phases of the basketball jump shot.

8. The computer program product of claim 7 wherein the phases comprise at least two members of the group consisting of (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase.

9. The computer program product of claim 7 wherein the phases comprise (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase.

10. The computer program product of claim 5, wherein the plurality of processor-executable instructions being further configured to:

compute an arc of a basketball for the basketball jump shot based on the video data.

11. The computer program product of claim 5, wherein the plurality of processor-executable instructions being further configured to:

compute data indicative of a release rate for the basketball jump shot based on the video data.

12. The computer program product of claim 5, wherein the plurality of processor-executable instructions being further configured to:

maintain data indicative of whether the basketball jump shot went through a basketball hoop.

13. The computer program product of claim 12, wherein the plurality of processor-executable instructions being further configured to:

repeat the method steps for a plurality of basketball jump shots by the person; and

build a database associated with the basketball jump shots, the database including the determined deviations for the basketball jump shots.

14. The computer program product of claim 13 wherein the database further associates a plurality of the basketball jump shots with the data indicative of whether the basketball jump shots went through the basketball hoop.

15. The computer program product of claim 14, wherein the plurality of processor-executable instructions being further configured to:

analyze the database to generate metrics about the person'"'"'s basketball jump shots; and

communicate the generate metrics for presentation to a user.

16. The computer program product of claim 14, wherein the plurality of processor-executable instructions being further configured to:

perform the frame determining step, the comparing step, and the deviation determining step with respect to a plurality of phases of the basketball jump shots, wherein the phases comprise (1) a base phase, (2) a frame phase, (3) a release phase, (4) a follow-through phase, and (5) a finish phase; and

wherein the computer program product further comprises instructions being configured, upon execution by a processor, to analyze the database to correlate the determined deviations for the basketball jump shots with respect to the data indicative of whether the basketball jump shots went through the basketball hoop.

17. The computer program product of claim 16, wherein the plurality of processor-executable instructions being further configured to:

identifying which determined deviations exhibit a correlation beyond a threshold with respect to basketball jump shots that did not go through the basketball hoop.

18. The computer program product of claim 1 wherein the frame determining step comprises:

the processor searching a plurality of the frames of the video data to identify a frame that exhibits a plurality of characteristics that are representative of the physical activity phase.

19. The computer program product of claim 1, wherein the plurality of processor-executable instructions being further configured to:

the processor communicating a result based on the determined deviation.

20. The computer program product of claim 1 wherein the comparing step comprises:

the processor determining the body position in the determined frame; and

the processor superimposing the reference skeleton over the determined body position.

21. The computer program product of claim 14, wherein the plurality of processor-executable instructions being further configured to:

the processor receiving user input indicative of body measurement data for the person; and

the processor defining the reference skeleton in response to user input such that the reference skeleton is scaled for the person'"'"'s body measurement data.

22. The computer program product of claim 1 wherein the video data exhibits a perspective with respect to the person.

23. The computer program product of claim 22 wherein the perspective comprises a member of the group consisting of a front view and a side view.

24. The computer program product of claim 1, wherein the plurality of processor-executable instructions being further configured to:

perform the method steps with respect to first video data from a first perspective and second video data from a second perspective.

25. The computer program product of claim 1 plurality of processor-executable instructions being further configured to:

capture the video data with a camera resident on a mobile computing device.

26. The computer program product of claim 1 wherein the determined frame comprises a plurality of frames that depict the person'"'"'s body position in the physical activity phase.

27. The computer program product of claim 1 wherein the processor comprises a plurality of processors.

28. The computer program product of claim 1, wherein the skeleton is a plurality of images comprising joints and lines connecting the joints.

29. The computer program product of claim 1, wherein the skeleton is a plurality of data points indicating the location of joints and limbs.

30. The computer program product of claim 1, wherein the plurality of processor-executable instructions being further configured to:

calculate body edge coordinates and a body size; and

determine joint coordinates based on the body edge coordinates and the body size.

31. The computer program product of claim 1, wherein the physical activity comprises one from the group comprising:

a volleyball serve, a tennis serve, throwing a football, kicking a football, punting a football, a golf swing, discus throwing, shot put throwing, high jumping, pole vaulting, hurdling, long jumping, swimming back stroke, swimming free style stroke, swimming butterfly stroke, and swimming breast stroke.

32. The computer program product of claim 1, wherein the physical activity comprises one from the group comprising:

performing an anterior cruciate ligament injury prevention jump test, an over-pronating detection running test, and an under-pronating detection running test.

33. A method for video analysis of a physical activity, the method comprising:

processing video data, the video data comprising a plurality of frames of a person performing the physical activity, wherein the processing step comprises;

determining a frame of the video data that depicts a body position of the person in a phase of the physical activity;

comparing the body position in the frame with a reference skeleton for the phase of the physical activity; and

determining a deviation between the frame body position and the reference skeleton based on the comparing step; and

wherein the frame determining, comparing, and deviation determining steps are performed by a processor.

34. An apparatus for video analysis of a physical activity, the apparatus comprising:

a processor configured for processing video data, the video data comprising a plurality of frames of a person performing the physical activity, wherein the processor is configured to;

determine a frame of the video data that depicts a body position of the person in a phase of the physical activity;

compare the body position in the frame with a reference skeleton for the phase of the physical activity; and

determine a deviation between the frame body position and the reference skeleton based on the comparison operation.

35. The apparatus of claim 34 wherein the processor is resident on a mobile computing device.

36. The apparatus of claim 34 wherein the processor is resident on a server system.

37. A computer program product comprising:

a plurality of processor-executable instructions, the instructions being resident on a non-transitory computer-readable storage medium of a computing device and being configured, upon execution by a processor, to;

capture video of a person performing the physical activity using a camera associated with a mobile device;

determine a first frame of the captured video that depicts the person positioned in a first phase of the physical activity by searching for characteristics identifying the first phase of the physical activity in the captured video;

compare the person'"'"'s body position in the first frame to a first reference skeleton corresponding to the first phase of the physical activity; and

determine whether the person demonstrated a derivation from the first reference skeleton in the first phase of the physical activity while performing the physical activity as a result of comparing the first frame to the first reference skeleton.

38. The computer program product of claim 37, further comprising:

determining a second frame of the captured video that depicts the person positioned in a second phase of the physical activity by searching for characteristics identifying the second phase of the physical activity in all frames of the captured video;

comparing the person'"'"'s body position in the second frame to a second reference skeleton corresponding to the second phase of the physical activity;

determining whether the person demonstrated a derivation from the second reference skeleton in the second phase of the physical activity while performing the physical activity as a result of comparing the second frame to the second reference skeleton,

wherein the method steps are performed by a processor of the mobile device.

39. The computer-program product of claim 37, wherein comparing the person'"'"'s body position in the first frame to a first reference skeleton corresponding to the first phase of the physical activity, comprises:

superimposing the first reference skeleton over the first frame and comparing the superimposed first reference skeleton to the first frame.

40. An apparatus for video analysis of a physical activity, the apparatus comprising:

a processor configured for processing video data, the video data comprising a plurality of frames of a person performing the physical activity, wherein the processor is configured to;

determine a frame of the video data that depicts a body position of the person in a phase of the physical activity;

compare the body position in the frame with a reference skeleton for the phase of the physical activity;

determine a deviation between the frame body position and the reference skeleton based on the comparison operation;

determine whether the basketball jump shot went through a basketball hoop;

a database for storing data, the database configured to;

maintain data indicative of whether the basketball jump shot went through a basketball hoop; and

maintain data indicative of one or more determined derivations between the frame body position and the reference skeleton.

41. The apparatus of claim 40, wherein the processor is further configured to analyze the database to determine which determined derivations between the frame body position and the reference skeleton resulted in a missed shot.

42. The apparatus of claim 40, wherein the processor is further configured to report the determined derivations between the frame body position and the reference skeleton that resulted in a missed shot to a coach'"'"'s user terminal.