True space tracking of axisymmetric object flight using diameter measurement

• US 8,948,457 B2
• Filed: 06/18/2013
• Issued: 02/03/2015
• Est. Priority Date: 04/03/2013
• Status: Active Grant

• Current Assignee: Pillar Vision Incorporated
• Original Assignee: Pillar Vision Incorporated
• Inventors: Marty, Alan W., Carter, John
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): MOYER, ANDREW M
• Application Number:
  US13/921,162
• Publication Number:
  US 20140301598A1
• Time in Patent Office: 595 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/51 Housings

Abstract

Methods and apparatus for determining a trajectory of a axisymmetric object in 3-D physical space using a digital camera which records 2-D image data are described. In particular, based upon i) a characteristic length of the axisymmetric object, ii) a physical position of the camera determined from sensors associated with the camera (e.g., accelerometers) and iii) captured 2-D digital images from the camera including a time at which each image is generated relative to one another, a position, a velocity vector and an acceleration vector can be determined in three dimensional physical space for axisymmetric object objects as a function of time. In one embodiment, the method and apparatus can be applied to determine the trajectories of objects in games which utilize axisymmetric object objects, such as basketball, baseball, bowling, golf, soccer, rugby or football.

Claims

1. A smart phone comprising:

a housing;

a wireless network interface;

a camera lens and a touch screen display coupled to an exterior of the housing;

a processor communicatively coupled to accelerometers and a digital image sensor disposed within an interior of the housing wherein the digital image sensor receives light which passes through the camera lens;

the processor configured to

     1) receive a plurality of 2-D digital images from the digital image sensor,

     2) output the plurality of 2-D digital images to the display,

     3) select a portion of the plurality of 2-D digital images wherein the portion is selected based upon at least whether an axisymmetric object in free flight used in a play of a game is identifiable,

     4) receive sensor data from the accelerometers which is used to determine an orientation of the camera lens in an earth reference frame;

     5) based upon the orientation of the camera lens in the earth reference frame, transform first pixel data in each of the portion of the plurality of 2-D digital images from a first reference frame associated with a current orientation of the camera lens to second pixel data associated with a second reference frame in which the camera lens is resting on a horizontal plane perpendicular to an axis through a gravitational center of the earth and a line drawn through a center of lens perpendicular to a surface of lens is at a center of the pixel data;

     6) determine a plurality of first pixel lengths across the axisymmetric object in multiple directions of the second pixel data in each of the portion of the plurality of 2-D digital images to account for motion blurring and light blooming effects;

     7) based upon the plurality of first pixel lengths, determine a first characteristic pixel length of the axisymmetric object in each of the portion of the plurality of 2-D digital images;

     8) based upon first characteristic pixel length, determine distances from the camera lens to the axisymmetric object in physical space in each of the portion of the plurality of 2-D digital images;

     9) determine a center of the axisymmetric object in the second pixel data in each of the 2-D digital images;

     10) determine a plurality of second pixel lengths which measure changes in the center of the axisymmetric object between each of the portion of the plurality of 2-D digital images;

     11) based upon the plurality of second pixel lengths and the distances from the camera lens, determine a 3-D trajectory of the axisymmetric object as a function of time in physical space,

     12) upload, via the wireless network interface, the 3-D trajectory to a remote server,

     13) select one of the plurality of 2-D digital images,

     14) render a path showing the 3-D trajectory of the axisymmetric object into the selected one of the plurality of 2-D digital images and

     15) output to the display a new 2-D digital image including the path of the 3-D trajectory.

2. The smart phone of claim 1, wherein the axisymmetric object is a spherically shaped object.

3. The smart phone of claim 1, wherein the processor is further configured to determine a second characteristic pixel length of the axisymmetric object perpendicular to the first characteristic pixel length in each of the portion of the plurality of 2-D digital images;

select one of the first characteristic pixel length or the second characteristic pixel length and based upon the selected one of the first characteristic pixel length or the second characteristic pixel length, determine the distance to the axisymmetric object.

4. The smart phone of claim 3, wherein the axisymmetric object is a football.

5. The smart phone of claim 3, wherein the axisymmetric object is a Frisbee.

6. The smart phone of claim 1, wherein the processor is further configured to identify one or more secondary objects associated with a play of the game captured in the plurality of the 2-D digital images.

7. The smart phone of claim 1, wherein the processor is further configured to output a first image of the plurality of 2-D images to the touch screen display, which includes a secondary object, and receive input from a touch screen display wherein the input from the touch screen display designates a portion of first image where the secondary object appears.

8. The smart phone of claim 7, wherein the processor is further configured to search in the designated portion of the first image to identify the secondary object.

9. The smart phone of claim 7, wherein, after the secondary object is identified, the processor is further configured to determine a second distance to secondary object from the camera lens.

10. The smart phone of claim 1, wherein the processor is further configure to receive a pixel resolution of the 2-D digital images.

11. The smart phone of claim 10, wherein the distances are determined based upon the received pixel resolution.

12. The smart phone of claim 10, wherein the processor is further configured to receive or retrieve a characteristic length of the axisymmetric object measured in units associated with 3-D physical space.

13. The smart phone of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while partially obscured.

14. The smart phone of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while being held in a person'"'"'s hands.

15. The smart phone of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while rolling along a surface or in contact with a surface.

16. The smart phone of claim 1, wherein the function of the time is based upon a time associated with each of the 2-D digital images.

17. The smart phone of claim 1, wherein the processor is further configured to receive a recording speed associated with the digital image sensor and to determine the function of the time based upon the recording speed.

18. The smart phone of claim 1, wherein the processor is further configured to determine a velocity vector and magnitude of the velocity vector as the function of time of the axisymmetric object along the 3-D trajectory.

19. The smart phone of claim 1, wherein processor is further configured to render one or more of a graphical representation of one or more of a playing field, playing field markings or a secondary object associated with the game in the new 2-D image.

20. The smart phone of claim 1, wherein only a first portion of a playing field, a first portion of a playing field markings or a first portion of a secondary object is captured in the one of the plurality of 2-D images and wherein the processor is further configured to render a second portion of the playing field, a second portion the playing field markings and/or a second portion of a secondary object in the new 2-D image such that the first portion and the second portion form a complete playing field, a complete playing field markings or a complete secondary object in the new 2-D image.

21. The smart phone of claim 1, wherein the processor is further configured to generate a simulation of the 3-D trajectory of the axisymmetric object using data determined from the 2-D digital images as initial conditions for the simulation.

22. The smart phone of claim 21, wherein the simulation is used to extrapolate the 3-D trajectory of the axisymmetric object beyond what is captured in the plurality of 2-D images.

23. The smart phone of claim 1, wherein information derived from the 3-D trajectory is used as input for a video game.

24. The smart phone of claim 1, wherein, based upon the 3-D trajectory, the processor is further configured to output, training recommendations for improving performance of a participant in the game played using the axisymmetric object.