Method of analysing a video of sports motion

• US 9,744,421 B2
• Filed: 06/27/2012
• Issued: 08/29/2017
• Est. Priority Date: 06/27/2011
• Status: Active Grant

• Current Assignee: Swing Profile Limited (Integrity Analysis Ltd.)
• Original Assignee: Swing Profile Limited (Integrity Analysis Ltd.)
• Inventors: Chan, Shun Heng
• Primary Examiner(s): LEGESSE, NINI F
• Application Number:
  US13/824,135
• Publication Number:
  US 20130178304A1
• Time in Patent Office: 1,889 Days
• Field of Search: 473218, 473200, 473266, 473269, 473270, 473409, 434247, 434252, 434257
• CPC Class Codes: A63B 69/36 for golf
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A method, apparatus, system and computer program product for analyzing video images of a sports motion is provided for identification of key motion positions within a video and automatic extraction of segments of a video containing a sports motion. Video data displaying a sports motion has data representative of a number of image frames. For a number of image frames, one or more frame difference measures are calculated between the image frame and another image frame. The frame difference measures are analyzed to identify a plurality of image frames that each show a key position of the sports motion. The segment of the video containing the sports motion showing the sports motion is between two of the image frames showing the key positions. Frame difference measures may be calculated based on pixel difference measures or optical flow techniques.

Claims

1. A computer-implemented method of analysing a video for improving sports training of a person, the method comprising:

receiving data representative of a number T of image frames constituting the video, wherein at least a portion of the video displays, in a single shot video sequence, the person performing, during the sports training, a sports motion, the sports motion being selected from a group consisting of;

striking a ball with a sports implement and simulating striking a ball with the sports implement, the video being captured from behind the person, and wherein during the sports motion the person adopts an impact position, the impact position being selected from a group consisting of;

the position of the person during the sports motion when impacting the ball with the sports implement and the position during the sports motion corresponding to when the person would have hit a ball with the sports implement;

calculating by a hardware processor, for each of a number T_C of the image frames, one or more frame difference measures between a respective image frame and a selected other image frame, wherein each image frame comprises a number P of pixels or groups of pixels and wherein each of the one or more frame difference measures comprises a measure of a difference between each of a number P_C of pixels or groups of pixels and each of the corresponding pixels or groups of pixels in the selected other image frame, wherein each of the one or more frame difference measures comprises a difference location indicator, the difference location indicator being a measure of the position of pixels or groups of pixels differing by at least a threshold amount between the respective image frame and the selected other image frame;

analysing by a hardware processor one of the one or more frame difference measures for each of the image frames to determine a downswing phase of the sports motion;

analysing the difference location indicators for each of the image frames to identify a peak in the difference location indicators in the determined downswing phase of the sports motion;

identifying an impact image frame of the number T of the image frames that shows the person adopting the impact position, the impact image frame corresponding to the image frame at a time of the peak in the difference location indicators in the determined downswing phase of the sports motion;

extracting impact image frame data representative of the impact image frame that shows the person adopting the impact position from the data representative of the number T of image frames constituting the video; and

outputting on a display device the impact image frame showing the person in the impact position, the impact image frame being output on the display device separately from the video for providing feedback for correcting training errors of the person during the sports training.

2. The computer-implemented method as claimed in claim 1, further comprising:

analysing the one or more frame difference measures to identify one or more other predetermined key positions of the person during the sports motion.

3. The computer-implemented method as claimed in claim 2, wherein the step of analysing the one or more frame difference measures to identify a plurality of image frames that each show the one or more other predetermined key positions of the person during the sports motion comprises:

analysing by a hardware processor one of the one or more frame difference measures for each of the image frames to identify a plurality of patterns in temporal variations in the one or more frame difference measures;

associating each pattern with a motion phase of the sports motion;

analysing by a hardware processor one of the one or more frame difference measures for each of the image frames to identify a sub-pattern in the one or more frame difference measures within the pattern associated with the respective motion phases of the sports motion;

identifying image frames of the number T of the image frames that shows the person adopting the one or more other predetermined key positions of the person during the sports motion, the identified image frames corresponding to the image frames associated with the identified sub-pattern in the one or more frame difference measures;

extracting image frame data representative of the one or more image frames that shows the person adopting the one or more other predetermined key positions of the person during the sports motion from the data representative of the number T of image frames constituting the video; and

outputting on the display device the one or more image frames showing the person adopting the one or more other predetermined key positions of the person during the sports motion, the one or more image frames being output on the display device separately from the video, for providing feedback for correcting training errors of the person during the sports training.

4. The computer-implemented method as claimed in claim 3, wherein the method comprises extracting a portion of the video in between two image frames showing the person adopting the one or more other predetermined key positions of the person during the sports motion and outputting said portion of the video on a display device separately from the video, for providing feedback for correcting training errors of the person during the sports training.

5. The computer-implemented method as claimed in claim 3, wherein the step of identifying a plurality of patterns comprises identifying peaks and/or troughs in temporal variations in the frame difference measures.

6. The computer-implemented method as claimed in claim 2, wherein the method comprises identifying each of the image frames showing the person in one of the predetermined key positions of the person during the sports motion using one of a plurality of rules, each rule being predetermined for the respective predetermined key position of the person.

7. The computer-implemented method as claimed in claim 1, wherein the one or more frame difference measures comprise one or more of the following:

a measure of the amount of movement between image frames;

a measure of the position of movement between image frames; and

a measure of the velocity or a derivative thereof of movement between image frames.

8. The computer-implemented method as claimed in claim 1, wherein the sports motion is a golf swing.

9. The computer-implemented method as claimed in claim 1, wherein the one or more frame difference measures comprises two or more frame difference measures and the two or more frame difference measures comprises a difference ratio DRatio(d_thres), the difference ratio DRatio(d_thres) being the proportion of pixels or groups of pixels in the respective image frame differing by at least a predetermined threshold from each of the corresponding pixels or groups of pixels in the selected other image frame.

10. The computer-implemented method as claimed in claim 9, wherein the method comprises analysing the difference ratios DRatio(d_thres) to determine the downswing phase of the sports motion, the downswing phase of the sports motion being when the difference ratios DRatio(d_thres) are relatively high compared to the difference ratios DRatio(d_thres) during other phases of the sports motion.

11. The computer-implemented method as claimed in claim 1, wherein the one or more frame difference measures comprises at least one of the group selected from:

a vertical difference location indicator being a measure of the vertical position of pixels or groups of pixels differing by at least a threshold amount between the respective image frame and the selected other image frame; and

a horizontal difference location indicator being a measure of the horizontal position of pixels or groups of pixels differing by at least a threshold amount between the respective image frame and the selected other image frame.

12. The computer-implemented method as claimed in claim 11, wherein the method comprises analysing at least one of:

the vertical difference location indicators; and

a velocity of pixel movement between the respective image frame and the selected other image frame, to determine the downswing phase of the sports motion.

13. The computer-implemented method as claimed in claim 12, wherein the method comprises calculating, for each of the number T_C of the image frames, a difference frame D between the respective image frame and a selected other image frame, wherein the difference frame D comprises one of the one or more frame difference measures between each of the number P_C of pixels or groups of pixels and each of the corresponding pixels or groups of pixels in the selected other image frame.

14. The computer-implemented method as claimed in claim 13, wherein the vertical difference location indicator is the p^th percentile of the vertical position of pixels or groups of pixels exceeding a predetermined threshold in the respective difference frame D.

15. The computer-implemented method as claimed in claim 1, wherein the method further comprises the step of identifying the portion of the video displaying the person while performing the sports motion.

16. The computer-implemented method as claimed in claim 1, wherein the method further comprises analysing the one or more frame difference measures and thereby identifying one or more motion phases other than the downswing phase of the sports motion.

17. The computer-implemented method as claimed in claim 1, wherein the method comprises analyzing, by a hardware processor, the impact image frame data representative of the impact image frame that shows the person adopting the impact position to automatically generate the feedback for correcting training errors of the person during the sports training and outputting the feedback on the display device.

18. The computer-implemented method as claimed in claim 17, wherein the step of analysing the impact image frame data representative of the impact image frame that shows the person adopting the impact position to generate the feedback for correcting training errors comprises:

executing a straight line detection algorithm to detect the sports implement in the impact image frame; and

outputting on the display device the impact image frame showing the person in the impact position while simultaneously displaying on the display device a straight line corresponding to the position of the sports implement in the impact image frame.

19. The computer-implemented method as claimed in claim 1, wherein the method comprises storing the impact image frame data representative of the impact image frame that shows the person adopting the impact position in a memory.

20. The computer-implemented method as claimed in claim 19, wherein the method comprises deleting data representative of the number T of image frames other than the stored impact image frame data representative of the impact image frame that shows the person adopting the impact position.

21. An apparatus for analysing a video for improving sports training of a person, the apparatus comprising:

a memory configured to store data representative of a number T of image frames constituting the video, wherein at least a portion of the video displays, in a single shot video sequence, the person performing, during the sports training, a sports motion, the sports motion being selected from a group consisting of;

striking a ball with a sports implement and simulating striking a ball with the sports implement, the video being captured from behind the person, and wherein during the sports motion the person adopts an impact position, the impact position being selected from a group consisting of;

the position of the person during the sports motion when impacting the ball with the sports implement and the position during the sports motion corresponding to when the person would have hit a ball with the sports implement;

a processor configured to calculate, for each of a number T_C of the image frames, one or more frame difference measures between a respective image frame and a selected other image frame, wherein each image frame comprises a number P of pixels or groups of pixels and wherein each of the one or more frame difference measures comprises a measure of a difference between each of a number P_C of pixels or groups of pixels and each of the corresponding pixels or groups of pixels in the selected other image frame, wherein each of the one or more frame difference measures comprises a difference location indicator, the difference location indicator being a measure of the position of pixels or groups of pixels differing by at least a threshold amount between the respective image frame and the selected other image frame, the processor being further configured to;

analyse one of the one or more frame difference measures to determine a downswing phase of the sports motion;

analyse the difference location indicators for each of the image frames to identify a peak in the difference location indicators in the determined downswing phase of the sports motion;

identify an impact image frame of the number T of the image frames that shows the person adopting the impact position, the impact image frame corresponding to the image frame at a time of the peak in the difference location indicators in the determined downswing phase of the sports motion;

extract impact image frame data representative of the impact image frame that shows the person adopting the impact position from the data representative of the number T of image frames constituting the video; and

output on a display device the impact image frame showing the person in the impact position, the impact image frame being output on the display device separately from the video for providing feedback for correcting training errors of the person during the sports training.

22. The apparatus as claimed in claim 21, further comprising:

a video capture device for recording the video.