Systems and methods for coordinating radar data and image data to track a flight of a projectile

• US 10,052,542 B2
• Filed: 02/22/2016
• Issued: 08/21/2018
• Est. Priority Date: 07/02/2004
• Status: Active Grant

• Current Assignee: Trackman A/S
• Original Assignee: Trackman A/S
• Inventors: Tuxen, Fredrik, Brink, Frederik Ettrup
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US15/050,456
• Publication Number:
  US 20160247292A1
• Time in Patent Office: 911 Days
• Field of Search: None
• CPC Class Codes: A63B 2024/0031 at the starting point
• CPC Class Codes: A63B 2024/0034 during flight
• CPC Class Codes: A63B 2071/0636 3D visualisation
• CPC Class Codes: A63B 2071/065 Visualisation of specific exercise parameters
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 2102/16 Table tennis
• CPC Class Codes: A63B 2102/18 Baseball, rounders or similar games
• CPC Class Codes: A63B 2102/20 Cricket
• CPC Class Codes: A63B 2102/32 Golf
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/12 Absolute positions, e.g. by using GPS
• CPC Class Codes: A63B 2220/20 Distances or displacements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/35 Spin
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2220/807 Photo cameras
• CPC Class Codes: A63B 2220/89 Field sensors, e.g. radar systems
• CPC Class Codes: A63B 2243/0025 Football American football A63B2243/007
• CPC Class Codes: A63B 2243/0033 Handball
• CPC Class Codes: A63B 2243/0037 Basketball
• CPC Class Codes: A63B 2243/0095 : Volleyball
• CPC Class Codes: A63B 24/0003 : Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 : Tracking a path or terminating locations
• CPC Class Codes: A63B 60/46 : Measurement devices associated with golf clubs, bats, rackets or the like for measuring physical parameters relating to sporting activity, e.g. baseball bats with impact indicators or bracelets for measuring the golf swing
• CPC Class Codes: A63B 69/36 : for golf
• CPC Class Codes: A63B 69/3623 : for driving A63B69/3608, A63B69/3658, A63B69/3661, A63B69/3667 and A63B69/3691 take precedence
• CPC Class Codes: A63B 69/3658 : Means associated with the ball for indicating or measuring, e.g. speed, direction
• CPC Class Codes: A63B 71/06 : Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: A63B 71/0605 : Decision makers and devices using detection means facilitating arbitration
• CPC Class Codes: A63B 71/0619 : Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
• CPC Class Codes: G06F 18/22 : Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10044 : Radar image
• CPC Class Codes: G06T 2207/30221 : Sports video; Sports image
• CPC Class Codes: G06T 2207/30224 : Ball; Puck
• CPC Class Codes: G06T 2207/30241 : Trajectory
• CPC Class Codes: G06T 3/02 : Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/246 : using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/32 : using correlation-based methods
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06T 7/74 : involving reference images or patches
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A system for coordinating radar data and image data to track a flight of a projectile includes a radar, an imager, and a controller. The imager provides an image of an area into which a projectile is to be launched. The controller receives the image and an identification of a target within the image and identifies a target line connecting the target and a launch position of the projectile. The controller also receives data from the radar relative to at least a portion of a trajectory of the projectile, generates image based trajectory data by correlating the radar data and the image, and alters the image to include the image based trajectory data within the image and the target line within the image.

Claims

1. A system for coordinating radar data and image data to track a flight of a projectile, comprising:

a radar;

an imager providing a plurality of images of an area into which a projectile is to be launched; and

a controller receiving the images and an identification of a target within a first one of the images, the controller identifying a target line connecting the target and a launch position of the projectile, the controller also receiving data from the radar relative to at least a portion of a trajectory of the projectile, the controller generating image based trajectory data by correlating the radar data and a second one of the images, the controller altering the second image to include the image based trajectory data within the second image and the target line within the second image,

wherein the controller is configured to position the target line in the second image based on an orientation and position of a radar coordinate system and an imager coordinate system.

2. The system according to claim 1, wherein the controller determines a deviation between the portion of the trajectory and the target line.

3. The system according to claim 1, wherein the target is identified by input received by the controller from a user.

4. The system according to claim 3, wherein the user input is generated by interaction with the first image.

5. The system according to claim 1, wherein the controller receives data corresponding to a distance from the imager to the target and determines based on the distance from the imager to the target and the data from the radar a deviation of the projectile between a direction of flight of the projectile and the target line.

6. The system according to claim 1, wherein the launch position of the projectile is within a field of view of the imager and a field of view of the radar, the controller determining the launch position based on the data from the radar.

7. The system according to claim 1, wherein the controller alters the second image while the projectile is in flight.

8. The system according to claim 1, wherein the controller determines a position of the portion of the trajectory within the second image and alters the second image to include a visual representation of the portion of the trajectory within the second image.

9. The system according to claim 8, wherein the controller alters the second image to include a visual representation of the target line.

10. A device for coordinating radar data and image data to track the flight of a projectile comprising a controller receiving a plurality of images and an identification of a target within a first one of the images, the controller identifying a target line connecting the target and a launch position of the projectile, the controller also receiving data from a radar relative to at least a portion of a trajectory of the projectile, the controller generating image based trajectory data by correlating the radar data and the images, the controller altering a second one of the images to include the image based trajectory data within the second image and the target line within the second image.

11. The device according to claim 10, wherein the controller comprises a computing device receiving data from a radar positioned so that the portion of the trajectory is within a field of view of the radar and wherein the computing device receives the images from an imager having a field of view including the target and the portion of the trajectory.

12. The device according to claim 11, wherein controller receives input identifying the target and identifies the target as an intersection of vertical and horizontal reference lines.

13. The device according to claim 10, wherein the controller determines a position of the portion of the trajectory within the second image and alters the second image to include a visual representation of the portion of the trajectory within the second image.

14. A method for coordinating radar data and image data to track a flight of a projectile, comprising the steps of:

receiving by a controller a plurality of images and an identification of a target within a first one of the images as well as radar data relating to at least a portion of a trajectory of a projectile traveling through a field of view represented by the first image;

identifying by the controller a target line connecting the target and a launch position of the projectile;

generating by the controller image based trajectory data by correlating the radar data and the images;

altering a second one of the images to include the image based trajectory data and the target line within the second image; and

manipulating, by the controller, the second image to position the target line in the altered second image based on an orientation and position of a radar coordinate system and an imager coordinate system.

15. The method according to claim 14, further comprising the step of determining a distance from the launch position to the target.

16. The method according to claim 14, wherein the controller correlates the radar data and the image based on data corresponding to a position of the imager relative to the radar.

17. The method according to claim 15, wherein the distance from the launch position to the target is input by a user.

18. The method according to claim 14, wherein the distance from the launch position to the target is determined by the controller based on a comparison of an angular extent of the target within one of the images and a known size of the target.

21. The method according to claim 14, wherein the controller determines a position of the portion of the trajectory within the image and alters the image to include a visual representation of the portion of the trajectory within the image.

19. A method for coordinating radar data and image data to track a flight of a projectile, comprising the steps of:

receiving by a controller an image and an identification of a target within the image as well as radar data relating to at least a portion of a trajectory of a projectile traveling through a field of view represented by the image;

identifying by the controller a target line connecting the target and a launch position of the projectile;

generating by the controller image based trajectory data by correlating the radar data and the image; and

altering the image to include the image based trajectory data and the target line within the image, wherein the controller identifies a radar coordinate system corresponding to the data received from the radar and an image coordinate system corresponding to the image and correlates the data from the radar with the image by coordinate translating the data from the radar to the image coordinate system through a three dimensional translation and rotation.

22. The method according to claim 19, wherein the controller determines a deviation between the portion of the trajectory and the target line.

20. A system for coordinating radar data and image data to track a flight of a projectile, comprising:

an imager producing a plurality of images of a desired field of view;

a radar tracking objects moving through at least a portion of the desired field of view;

a controller receiving the images and an identification of a target within a first one of the images;

the controller receiving data from the radar relating to at least a portion of a trajectory of a projectile traveling through the desired field of view, the controller identifying a target line connecting the target and a launch position of the projectile and generating image based trajectory data by correlating the radar data and at least a first one of the images, the controller altering a second one of the images to include the image based trajectory data and the target line, wherein the controller identifies a radar coordinate system corresponding to the data received from the radar and an image coordinate system corresponding to the images and correlates the data from the radar with the image by coordinate translating the data from the radar to the image coordinate system through a three dimensional translation and rotation.