Systems and methods for indicating user performance in launching a basketball toward a basketball hoop

• US 10,762,642 B2
• Filed: 01/30/2020
• Issued: 09/01/2020
• Est. Priority Date: 04/03/2013
• Status: Active Grant

• Current Assignee: Pillar Vision Incorporated
• Original Assignee: Pillar Vision Incorporated
• Inventors: Marty, Alan W., Carter, John
• Primary Examiner(s): Moyer, Andrew M
• Application Number:
  US16/777,838
• Publication Number:
  US 20200167936A1
• Time in Patent Office: 215 Days
• Field of Search: 382103, 382154
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/51 Housings

Abstract

Methods and apparatus for determining a trajectory of a axisymmetric object in 3-D physical space using a digital camera which records 2-D image data are described. In particular, based upon i) a characteristic length of the axisymmetric object, ii) a physical position of the camera determined from sensors associated with the camera (e.g., accelerometers) and iii) captured 2-D digital images from the camera including a time at which each image is generated relative to one another, a position, a velocity vector and an acceleration vector can be determined in three dimensional physical space for axisymmetric object objects as a function of time. In one embodiment, the method and apparatus can be applied to determine the trajectories of objects in games which utilize axisymmetric object objects, such as basketball, baseball, bowling, golf, soccer, rugby or football.

Claims

1. A mobile computing device for use in basketball, comprising

a digital camera for capturing a plurality of two-dimensional (2D) images;

and

at least one processor programmed with instructions that, when executed by the at least one processor, cause the at least one processor to;

identify a basketball hoop within the plurality of 2D images based on a shape of the basketball hoop within the plurality of 2D images;

determine a position of the basketball hoop in three-dimensional (3D) space based on at least one of the identified basketball hoop and a backboard coupled to the identified basketball hoop;

identify a secondary object in the plurality of 2D images;

determine a position of the secondary object in 3D space based on at least a first 2D image of the plurality of 2D images;

identify a basketball within the plurality of 2D images;

when the basketball is launched toward the basketball hoop, determine a plurality of 2D locations of the basketball along a trajectory of the basketball toward the basketball hoop based on the plurality of 2D images;

determine a position of the basketball in 3D space for at least one of the 2D locations based on the determined position of the secondary object in 3D space and a location of the basketball relative to the secondary object in at least a second 2D image of the plurality of 2D images;

determine a trajectory of the basketball in 3D space based on the plurality of 2D locations, the position of the basketball hoop in 3D space, and the position of the basketball in 3D space;

determine an orientation of the digital camera in 3D space based on at least one of the shape of the basketball hoop and the backboard within the plurality of 2D images; and

based on the determined trajectory, provide feedback indicative of a performance of a user in launching the basketball toward the basketball hoop.

2. The mobile computing device of claim 1, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine the position of the basketball hoop in 3D space based on a size of the basketball hoop within the plurality of 2D images.

3. The mobile computing device of claim 1, wherein the mobile communication device is a smart phone.

4. The mobile computing device of claim 1, wherein the mobile communication device is a tablet computer.

5. The mobile computing device of claim 1, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine, based on the determined trajectory, an angle at which the basketball is launched along the trajectory, and wherein the feedback indicates the angle.

6. The mobile computing device of claim 1, wherein determination of the position of the secondary object in 3D space is based on a size of the secondary object in the plurality of 2D images and a size for the secondary object specified by a user input.

7. The mobile computing device of claim 1, wherein the secondary object is a marking on a basketball court for the basketball hoop.

8. The mobile computing device of claim 1, wherein the secondary object is the user.

9. The mobile computing device of claim 1, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine the orientation of the digital camera in 3D space based on the shape of the basketball hoop within the plurality of 2D images.

10. The mobile computing device of claim 9, wherein the instructions, when executed by the at least one processor, cause the at least one processor to determine the position of the basketball hoop in 3D space based on a size of the basketball hoop within the plurality of 2D images.

11. A mobile computing device for use in basketball, comprising

a digital camera for capturing a plurality of two-dimensional (2D) images;

and

at least one processor programmed with instructions that, when executed by the at least one processor, cause the at least one processor to;

identify a basketball hoop within the plurality of 2D images based on a shape of the basketball hoop within the plurality of 2D images;

determine a distance to the digital camera of the basketball hoop or a backboard coupled to the basketball hoop within the plurality of 2D images;

determine an orientation of the basketball hoop to the digital camera based on at least one of the shape of the basketball hoop and the backboard within the plurality of 2D images;

determine a position of the basketball hoop in three-dimensional (3D) space based on the identified basketball hoop, the determined distance, and the determined orientation;

identify a basketball within the plurality of 2D images;

when the basketball is launched toward the basketball hoop, determine a plurality of 2D locations of the basketball along a trajectory of the basketball toward the basketball hoop;

determine a trajectory of the basketball in 3D space based on the plurality of 2D locations and the position of the basketball hoop in 3D space; and

based on the determined trajectory, provide feedback indicative of performance of a user in launching the basketball toward the basketball hoop.

12. The mobile computing device of claim 11, wherein the determined distance is of the basketball hoop to the digital camera and is based on a size of the basketball hoop within the plurality of 2D images.

13. The mobile computing device of claim 11, wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

identify a secondary object in the plurality of 2D images;

determine a position of the secondary object in 3D space;

determine a position of the basketball in 3D space for at least one of the 2D locations based on the determined position of the secondary object in 3D space and a location of the basketball relative to the secondary object in the plurality of 2D images; and

determine the trajectory of the basketball in (3D) space based on the position of the basketball in 3D space.

14. The mobile computing device of claim 13, wherein determination of the position of the secondary object in 3D space is based on a size of the secondary object in the plurality of 2D images and a size for the secondary object specified by a user input.

15. The mobile computing device of claim 13, wherein the secondary object is a marking on a basketball court for the basketball hoop.

16. The mobile computing device of claim 13, wherein the secondary object is the user.

17. A method for use in basketball, comprising:

capturing a plurality of two-dimensional (2D) images with a digital camera;

identifying a basketball hoop within the plurality of 2D images with at least one processor based on a shape of the basketball hoop within the plurality of 2D images;

determining, with the at least one processor, a position of the basketball hoop in three-dimensional (3D) space based on at least one of the identified basketball hoop and a backboard coupled to the identified basketball hoop;

identifying, with the at least one processor, a secondary object in the plurality of 2D images;

determining, with the at least one processor, a position of the secondary object in 3D space based on at least a first 2D image of the plurality of 2D images;

identifying, with the at least one processor, a basketball within the plurality of 2D images;

when the basketball is launched toward the basketball hoop, determining with the at least one processor a plurality of 2D locations of the basketball along a trajectory of the basketball toward the basketball hoop based on the plurality of 2D images;

determining, with the at least one processor, a position of the basketball in 3D space for at least one of the 2D locations based on the determined position of the secondary object in 3D space and a location of the basketball relative to the secondary object in at least a second 2D image of the plurality of 2D images;

determining, with the at least one processor, a trajectory of the basketball in 3D space based on the plurality of 2D locations, the position of the basketball hoop in 3D space, and the position of the basketball in 3D space;

determining, with the at least one processor, an orientation of the digital camera in 3D space based on at least one of the shape of the basketball hoop and the backboard within the plurality of 2D images; and

based on the determined trajectory, providing feedback to a user, the feedback indicative of a performance of the user in launching the basketball toward the basketball hoop.

18. The method of claim 17, further comprising determining, with the at least one processor, the position of the basketball hoop in 3D space based on a size of the basketball hoop within the plurality of 2D images.

19. The method of claim 17, further comprising determining, with the at least one processor based on the determined trajectory, an angle at which the basketball is launched along the trajectory, wherein the feedback indicates the angle.

20. The method of claim 17, further comprising receiving, with the at least one processor, a user input indicating a size of the secondary object, wherein the determining the position of the secondary object in 3D space is based on a size of the secondary object in the plurality of 2D images and the size of the secondary object indicated by the user input.

21. The method of claim 17, wherein the secondary object is a marking on a basketball court for the basketball hoop.

22. The method of claim 17, wherein the secondary object is the user.

23. The method of claim 17, further comprising determining, with the at least one processor, the orientation of the digital camera in 3D space based on the shape of the basketball hoop within the plurality of 2D images.

24. The method of claim 23, wherein the determining the position of the basketball hoop in 3D space is based on a size of the basketball hoop within the plurality of 2D images.

25. A method, comprising

capturing a plurality of two-dimensional (2D) images with a digital camera;

identifying, with at least one processor, a basketball hoop within the plurality of 2D images based on a shape of the basketball hoop within the plurality of 2D images;

determining, with the at least one processor, a distance to the digital camera of the basketball hoop or a backboard coupled to the basketball hoop within the plurality of 2D images;

determining, with the at least one processor, an orientation of the basketball hoop to the digital camera based on at least one of the shape of the basketball hoop and the backboard within the plurality of 2D images;

determining, with the at least one processor, a position of the basketball hoop in three-dimensional (3D) space based on the identified basketball hoop, the determined distance, and the determined orientation;

identifying, with the at least one processor, a basketball within the plurality of 2D images;

when the basketball is launched toward the basketball hoop, determining with the at least one processor a plurality of 2D locations of the basketball along a trajectory of the basketball toward the basketball hoop;

determining, with the at least one processor, a trajectory of the basketball in 3D space based on the plurality of 2D locations and the position of the basketball hoop in 3D space; and

based on the determined trajectory, providing feedback to a user, the feedback indicative of performance of the user in launching the basketball toward the basketball hoop.

26. The method of claim 25, wherein the determined distance is of the basketball hoop to the digital camera and is based on a size of the basketball hoop within the plurality of 2D images.

27. The method of claim 25, further comprising:

identifying, with the at least one processor, a secondary object in the plurality of 2D images;

determining, with the at least one processor, a position of the secondary object in 3D space;

determining, with the at least one processor, a position of the basketball in 3D space for at least one of the 2D locations based on the determined position of the secondary object in 3D space and a location of the basketball relative to the secondary object in the plurality of 2D images; and

determining, with the at least one processor, the trajectory of the basketball in (3D) space based on the position of the basketball in 3D space.

28. The method of claim 25, further comprising receiving, with the at least one processor, a user input indicating a size of the secondary object, wherein the determining the position of the secondary object in 3D space is based on a size of the secondary object in the plurality of 2D images and the size of the secondary object indicated by the user input.

29. The method of claim 25, wherein the secondary object is a marking on a basketball court for the basketball hoop.

30. The method of claim 25, wherein the secondary object is the user.