TRUE SPACE TRACKING OF AXISYMMETRIC OBJECT FLIGHT USING DIAMETER MEASUREMENT

• US 20150103168A1
• Filed: 12/22/2014
• Published: 04/16/2015
• Est. Priority Date: 04/03/2013
• Status: Active Grant

• Current Assignee: Pillar Vision Incorporated
• Original Assignee: Pillar Vision Incorporated
• Inventors: Carter, John, MARTY, Alan W.
• Granted Patent: US 9,697,617 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/51 Housings

Abstract

Methods and apparatus for determining a trajectory of a axisymmetric object in 3-D physical space using a digital camera which records 2-D image data are described. In particular, based upon i) a characteristic length of the axisymmetric object, ii) a physical position of the camera determined from sensors associated with the camera (e.g., accelerometers) and iii) captured 2-D digital images from the camera including a time at which each image is generated relative to one another, a position, a velocity vector and an acceleration vector can be determined in three dimensional physical space for axisymmetric object objects as a function of time. In one embodiment, the method and apparatus can be applied to determine the trajectories of objects in games which utilize axisymmetric object objects, such as basketball, baseball, bowling, golf, soccer, rugby or football.

Claims

1. An electronic device comprising:

a housing;

a wireless network interface;

a camera lens coupled to an exterior of the housing;

an input interface; and

a processor communicatively coupled to the input interface and a digital image sensor disposed within an interior of the housing wherein the digital image sensor receives light which passes through the camera lens;

the processor configured to

     1) receive a plurality of 2-D digital images from the digital image sensor,

     2) output the plurality of 2-D digital images to the display,

     3) select a portion of the plurality of 2-D digital images wherein the portion is selected based upon at least whether an axisymmetric object in free flight used in a play of a game is identifiable,

     4) receive input data from the input mechanism which is used to determine an orientation of the camera lens in a first reference frame;

     5) based upon the orientation of the camera lens in the first reference frame, transform first pixel data in each of the portion of the plurality of 2-D digital images from the first reference frame associated with a current orientation of the camera lens to second pixel data associated with a second reference frame;

     6) determine a plurality of first pixel lengths across the axisymmetric object in multiple directions of the second pixel data in each of the portion of the plurality of 2-D digital images to account for motion blurring and light blooming effects;

     7) based upon the plurality of first pixel lengths, determine a first characteristic pixel length of the axisymmetric object in each of the portion of the plurality of 2-D digital images;

     8) based upon first characteristic pixel length, determine distances from the camera lens to the axisymmetric object in physical space in each of the portion of the plurality of 2-D digital images;

     9) determine a center of the axisymmetric object in the second pixel data in each of the 2-D digital images;

     10) determine a plurality of second pixel lengths which measure changes in the center of the axisymmetric object between each of the portion of the plurality of 2-D digital images;

     11) based upon the plurality of second pixel lengths and the distances from the camera lens, determine a 3-D trajectory of the axisymmetric object as a function of time in physical space,

     12) select one of the plurality of 2-D digital images,

     13) render a path showing the 3-D trajectory of the axisymmetric object into the selected one of the plurality of 2-D digital images and

     14) output to the display a new 2-D digital image including the path of the 3-D trajectory.

2. The electronic device of claim 1, wherein the axisymmetric object is a spherically shaped object.

3. The electronic device of claim 1, wherein the processor is further configured to determine a second characteristic pixel length of the axisymmetric object perpendicular to the first characteristic pixel length in each of the portion of the plurality of 2-D digital images;

select one of the first characteristic pixel length or the second characteristic pixel length and based upon the selected one of the first characteristic pixel length or the second characteristic pixel length, determine the distance to the axisymmetric object.

4. The electronic device of claim 3, wherein the axisymmetric object is a football.

5. The electronic device of claim 3, wherein the axisymmetric object is a frisbee.

6. The electronic device of claim 1, wherein the processor is further configured to identify one or more secondary objects associated with a play of the game captured in the plurality of the 2-D digital images.

7. The electronic device of claim 1, wherein the processor is further configured to output a first image of the plurality of 2-D images to the touch screen display, which includes a secondary object, and receive input from a touch screen display wherein the input from the touch screen display designates a portion of first image where the secondary object appears.

8. The electronic device of claim 7, wherein the processor is further configured to search in the designated portion of the first image to identify the secondary object.

9. The electronic device of claim 7, wherein, after the secondary object is identified, the processor is further configured to determine a second distance to secondary object from the camera lens.

10. The electronic device of claim 1, wherein the processor is further configure to receive a pixel resolution of the 2-D digital images.

11. The electronic device of claim 10, wherein the distances are determined based upon the received pixel resolution.

12. The electronic device of claim 10, wherein the processor is further configured to receive or retrieve a characteristic length of the axisymmetric object measured in units associated with 3-D physical space.

13. The electronic device of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while partially obscured.

14. The electronic device of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while being held in a person'"'"'s hands.

15. The electronic device of claim 1, wherein the axisymmetric object in one of the 2-D digital images is identified while rolling along a surface or in contact with a surface.

16. The electronic device of claim 1, wherein the function of the time is based upon a time associated with each of the 2-D digital images.

17. The electronic device of claim 1, wherein the processor is further configured to receive a recording speed associated with the digital image sensor and to determine the function of the time based upon the recording speed.

18. The electronic device of claim 1, wherein the processor is further configured to determine a velocity vector and magnitude of the velocity vector as the function of time of the axisymmetric object along the 3-D trajectory.

19. The electronic device of claim 1, wherein processor is further configured to render one or more of a graphical representation of one or more of a playing field, playing field markings or a secondary object associated with the game in the new 2-D image.

20. The electronic device of claim 1, wherein only a first portion of a playing field, a first portion of a playing field markings or a first portion of a secondary object is captured in the one of the plurality of 2-D images and wherein the processor is further configured to render a second portion of the playing field, a second portion the playing field markings and/or a second portion of a secondary object in the new 2-D image such that the first portion and the second portion form a complete playing field, a complete playing field markings or a complete secondary object in the new 2-D image.

21. The electronic device of claim 1, wherein the processor is further configured to generate a simulation of the 3-D trajectory of the axisymmetric object using data determined from the 2-D digital images as initial conditions for the simulation.

22. The electronic device of claim 21, wherein the simulation is used to extrapolate the 3-D trajectory of the axisymmetric object beyond what is captured in the plurality of 2-D images.

23. The electronic device of claim 1, wherein information derived from the 3-D trajectory is used as input for a video game.

24. The electronic device of claim 1, wherein, based upon the 3-D trajectory, the processor is further configured to output, training recommendations for improving performance of a participant in the game played using the axisymmetric object.

25. The electronic device of claim 1, further comprising a bracket coupled to the housing configured to mount the housing to a wall.

26. The electronic device of claim 1, wherein the electronic device is one of a tablet computer or a phone.

27. An electronic device comprising:

a housing;

a wireless network interface;

a camera lens coupled to an exterior of the housing;

an input interface; and

a processor communicatively coupled to the input interface and a digital image sensor disposed within an interior of the housing wherein the digital image sensor receives light which passes through the camera lens;

the processor configured to

     1) receive a plurality of 2-D digital images from the digital image sensor,

     2) output the plurality of 2-D digital images to the display,

     3) select a portion of the plurality of 2-D digital images wherein the portion is selected based upon at least whether a football in free flight is identifiable,

     4) receive input data from the input mechanism which is used to determine an orientation of the camera lens in a first reference frame;

     5) based upon the orientation of the camera lens in the first reference frame, transform first pixel data in each of the portion of the plurality of 2-D digital images from the first reference frame associated with a current orientation of the camera lens to second pixel data associated with a second reference frame;

     6) determine a plurality of first pixel lengths across the football in multiple directions of the second pixel data in each of the portion of the plurality of 2-D digital images to account for motion blurring and light blooming effects;

     7) based upon the plurality of first pixel lengths, determine a first characteristic pixel length of the football in each of the portion of the plurality of 2-D digital images;

     8) based upon first characteristic pixel length, determine distances from the camera lens to the football in physical space in each of the portion of the plurality of 2-D digital images;

     9) determine a position associated with the football in the second pixel data in each of the 2-D digital images;

     10) determine a plurality of second pixel lengths which measure changes in the position of the football between each of the portion of the plurality of 2-D digital images;

     11) based upon the plurality of second pixel lengths and the distances from the camera lens, determine a 3-D trajectory of the football as a function of time in physical space,

     12) select one of the plurality of 2-D digital images,

     13) render a path showing the 3-D trajectory of the football into the selected one of the plurality of 2-D digital images and

     14) output to the display a new 2-D digital image including the path of the 3-D trajectory.

28. An electronic device comprising:

a housing;

a wireless network interface;

a camera lens coupled to an exterior of the housing;

an input interface; and

a processor communicatively coupled to the input interface and a digital image sensor disposed within an interior of the housing wherein the digital image sensor receives light which passes through the camera lens;

the processor configured to

     1) receive a plurality of 2-D digital images from the digital image sensor,

     2) output the plurality of 2-D digital images to the display,

     3) select a portion of the plurality of 2-D digital images wherein the portion is selected based upon at least whether a basketball in free flight is identifiable,

     4) receive input data from the input mechanism which is used to determine an orientation of the camera lens in a first reference frame;

     5) based upon the orientation of the camera lens in the first reference frame, transform first pixel data in each of the portion of the plurality of 2-D digital images from the first reference frame associated with a current orientation of the camera lens to second pixel data associated with a second reference frame;

     6) determine a plurality of first pixel lengths across the basketball in multiple directions of the second pixel data in each of the portion of the plurality of 2-D digital images to account for motion blurring and light blooming effects;

     7) based upon the plurality of first pixel lengths, determine a first characteristic pixel length of the basketball in each of the portion of the plurality of 2-D digital images;

     8) based upon first characteristic pixel length, determine distances from the camera lens to the basketball in physical space in each of the portion of the plurality of 2-D digital images;

     9) determine a position associated with the basketball in the second pixel data in each of the 2-D digital images;

     10) determine a plurality of second pixel lengths which measure changes in the position of the basketball between each of the portion of the plurality of 2-D digital images;

     11) based upon the plurality of second pixel lengths and the distances from the camera lens, determine a 3-D trajectory of the basketball as a function of time in physical space,

     12) select one of the plurality of 2-D digital images,

     13) render a path showing the 3-D trajectory of the basketball into the selected one of the plurality of 2-D digital images and

     14) output to the display a new 2-D digital image including the path of the 3-D trajectory.