METHODS AND SYSTEMS FOR BALL GAME ANALYTICS WITH A MOBILE DEVICE

• US 20190087661A1
• Filed: 08/23/2018
• Published: 03/21/2019
• Est. Priority Date: 09/21/2017
• Status: Active Grant

• Current Assignee: NEX Team Inc.
• Original Assignee: NEX Team Inc.
• Inventors: Lee, Keng Fai, Lam, Tin Yin, Chan, Wing Hung, Zhang, Qi, Sung, On Loy, Chan, Man Hon
• Granted Patent: US 10,489,656 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06Q 50/00 Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism healthcare informatics G16H
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel

Abstract

Methods and systems for ball shot attempt detection and game analytics generation are disclosed. The methods and systems perform steps to receive an input video of a ball gameplay, wherein the input video was captured using a stationary camera, and wherein frames of the input video comprises a goal; identify a Region of Interest (ROI) surrounding the goal by performing a first computer vision algorithm on the input video; detect a ball within the ROI during a shot attempt and determining a trajectory of the ball by performing a second computer vision algorithm on the input video; and identify a player relevant to the shot attempt based on the trajectory of the ball. In some embodiments, the computer vision algorithms comprise a convolution neural network (CNN). The present invention uses computer vision techniques to enable a resource-limited mobile device such as a smartphone to efficiently execute the new process.

Claims

1. A method for detecting a ball shot attempt captured by a single stationary camera, comprising:

receiving an input video captured using the single stationary camera, wherein frames of the input video comprise a goal;

detecting the goal, using a Convolutional Neural Network (CNN) module, by detecting more than one key feature points of the goal from the input video, wherein the CNN module has been trained using one or more prior input videos;

identifying a ball tracking Region of Interest (ROI) surrounding the goal;

detecting a ball within the ball tracking ROI over the frames of the input video;

determining a trajectory of the ball over the frames of the input video; and

detecting a ball shot attempt by analyzing the trajectory of the ball relative to the goal.

2. The method of claim 1, further comprising:

identifying a player relevant to the shot attempt based on the trajectory of the ball; and

detecting a foot location of the player before the shot is taken.

3. The method of claim 1, wherein the input video is captured in real-time.

4. The method of claim 1, wherein the determining of the trajectory of the ball is by backtracking the ball over a time duration.

6. The method of claim 4, wherein the input video over the time duration is down-sampled.

2-1. The method of claim 1, further comprising:

identifying a player relevant to the ball shot attempt based on the trajectory of the ball; and

detecting a foot location of the player before the ball shot attempt is taken.

3-1. The method of claim 1, wherein the input video is captured in real-time.

4-1. The method of claim 1, wherein the determining of the trajectory of the ball is by backtracking the ball over a time duration.

5-1. The method of claim 4, wherein the time duration is 2 seconds.

6-1. The method of claim 4, wherein the input video over the time duration is down-sampled.

7. The method of claim 1, wherein the trajectory of the ball is selected from the group consisting of a two-dimensional trajectory and a three-dimensional trajectory.

10. The method of claim 1, wherein the input video comprises a video of a basketball game, and wherein the method further comprises identifying four point-boundaries of a basketball court.

11. The method of claim 10, further comprising:

identifying a center of a free-throw line.

12. The method of claim 1, further comprising:

determining at least one shot attempt result.

13. The method of claim 12, further comprising:

generating a game analytics based on the at least one shot attempt result.

14. The method of claim 1, further comprising:

identifying a player relevant to the ball shot attempt based on the trajectory of the ball;

detecting a foot location of the player before the ball shot attempt is taken;

determining at least one shot attempt result; and

generating a game analytics based on the at least one shot attempt result and the foot location.

1-1. A method for detecting a ball shot attempt captured by a single mobile device camera, comprising:

receiving an input video captured using the single mobile device camera, wherein frames of the input video comprise a goal;

detecting the goal, using a Convolutional Neural Network (CNN) module, by detecting one or more key feature points of the goal from the input video, wherein the CNN module has been trained using one or more prior input videos,

identifying a ball tracking Region of Interest (ROI) surrounding the goal;

detecting a ball within the ball tracking ROI over the frames of the input video;

determining a trajectory of the ball over the frames of the input video; and

detecting the ball shot attempt based on the trajectory of the ball relative to the goal.

5. (canceled)

8. (canceled)

9. (canceled)

15. A system for detecting a ball shot attempt, comprising:

at least one processor; and

a non-transitory physical medium for storing program code and accessible by the processor, the program code when executed by the processor causes the processor to;

receive an input video captured using a single mobile device camera, wherein frames of the input video comprise a goal;

detect the goal, using a Convolutional Neural Network (CNN) module, by detecting one or more key feature points of the goal from the input video, wherein the CNN module has been trained using one or more prior input videos;

identify a ball tracking Region of Interest (ROI) surrounding the goal;

detect a ball within the ball tracking ROI over the frames of the input video;

determine a trajectory of the ball over the frames of the input video; and

detect the ball shot attempt based on the trajectory of the ball relative to the goal.

16. The system of claim 15, further comprising program code to:

identify a player relevant to the ball shot attempt based on the trajectory of the ball; and

detect a foot location of the player before the ball shot attempt is taken.

17. The system of claim 15, wherein the input video is captured in real-time.

18. The system of claim 15, wherein the determination of the trajectory of the ball is by backtracking the ball over a time duration.

19. The system of claim 15, wherein the input video comprises a video of a basketball game, and wherein the program code when executed by the processor further causes the processor to identify four point-boundaries of a basketball court.

20. A non-transitory computer-readable storage medium for detecting a ball shot attempt, the storage medium comprising program code stored thereon, the program code when executed by a processor causes the processor to:

receive an input video captured using a single mobile device camera, wherein frames of the input video comprise a goal;

detect the goal, using a Convolutional Neural Network (CNN) module, by detecting one or more key feature points of the goal from the input video, wherein the CNN module has been trained using one or more prior input videos;

identify a ball tracking Region of Interest (ROI) surrounding the goal;

detect a ball within the ball tracking ROI over the frames of the input video;

determine a trajectory of the ball over the frames of the input video; and

detect the ball shot attempt based on the trajectory of the ball relative to the goal.