Golf simulation system with reflective projectile marking

• US 9,199,153 B2
• Filed: 10/07/2009
• Issued: 12/01/2015
• Est. Priority Date: 07/30/2003
• Status: Active Grant

• Current Assignee: Interactive Sports Technologies Inc.
• Original Assignee: Interactive Sports Technologies Inc.
• Inventors: Dawe, Wayne, Zhao, Zuqiang
• Primary Examiner(s): Torrente, Richard
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US13/123,321
• Publication Number:
  US 20110273562A1
• Time in Patent Office: 2,246 Days
• Field of Search: 348/139
• US Class Current: 1/1
• CPC Class Codes: A63B 2024/0028 Tracking the path of an object, e.g. a ball inside a soccer pitch
• CPC Class Codes: A63B 2024/0031 at the starting point
• CPC Class Codes: A63B 2024/0034 during flight
• CPC Class Codes: A63B 2071/0636 3D visualisation
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/13 Relative positions
• CPC Class Codes: A63B 2220/20 Distances or displacements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/35 Spin
• CPC Class Codes: A63B 2220/62 Time or time measurement used for time reference, time stamp, master time or clock signal
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2220/807 Photo cameras
• CPC Class Codes: A63B 2225/74 with powered illuminating means, e.g. lights
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 47/025 Installations continuously collecting balls from the playing areas, e.g. by gravity, with conveyor belts
• CPC Class Codes: A63B 69/36 for golf
• CPC Class Codes: A63B 69/3658 Means associated with the ball for indicating or measuring, e.g. speed, direction
• CPC Class Codes: A63B 71/06 Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/047 : using sets of wires, e.g. crossed wires
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30224 : Ball; Puck
• CPC Class Codes: G06T 2207/30241 : Trajectory
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G09B 19/0038 : Sports
• CPC Class Codes: G09B 9/00 : Simulators for teaching or training purposes for the use of weapons F41; computing aspects G06; protocols for games, networked simulations or virtual reality H04L67/131
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A sports simulation system (100) comprises at least two imaging devices (128) capturing images of a projectile tracking region disposed in front of a display surface (124) from different vantages to detect a launched projectile traveling through the projectile tracking region towards the display surface; a projectile spin sensing unit (105) capturing images of a region at least partially overlapping with the projectile tracking region, each captured image comprising a projectile trail representing a travel path of the projectile when a projectile is present in the region during image capture; and at least one processing stage (104) receiving data from the imaging devices (128) and the projectile spin sensing unit (105) and determining the three-dimensional positions, velocity, acceleration and spin of a detected launched projectile traveling through the projectile tracking region, the three-dimensional positions, velocity, acceleration and spin being used by the at least one processing stage to calculate a trajectory of the launched projectile into a presented sports scene.

Claims

1. A sports simulation system comprising:

a projectile tracking apparatus comprising at least two imaging devices configured to capture images of a projectile tracking region disposed in front of a display surface from different vantages to detect a launched, spinning projectile traveling from a projectile launch area and through said projectile tracking region towards said display surface;

an overhead projectile spin sensing unit positioned intermediate said projectile launch area and said display surface and configured to capture images of a region adjacent said projectile launch area that at least partially overlaps with said projectile tracking region, each captured image comprising a projectile trail that comprises spaced apart high intensity regions resulting from an illuminated discrete elongate reflective marking on said projectile and representing a travel path of said projectile when a projectile is present in said region during image capture; and

at least one processing stage configured to receive image data from the imaging devices and said projectile spin sensing unit and to process the received image data to determine the three-dimensional positions, velocity, acceleration and spin rate and tilt of a detected launched, spinning projectile traveling through said projectile tracking region, the three-dimensional positions, velocity, acceleration and spin rate and tilt being used by said at least one processing stage to calculate a trajectory of said launched projectile into a sports scene presented on said display surface, wherein during processing of image data from said projectile spin sensing unit, said at least one processing stage is configured to generate a profile of the projectile trail over a plurality of images, determine the projectile trail length per image of said plurality of images, threshold each image to identify the high intensity regions corresponding to the elongate marking on said projectile, determine the distance between the high intensity regions in each image, calculate the spin rate of the projectile using the determined distance, the projectile trail length and the projectile spin sensing unit image frame rate and estimate a spin tilt axis of the projectile using the orientation of the high intensity regions in each image and the relative angle between the longitudinal axis of the high intensity regions and the longitudinal axis of the projectile trail.

2. A sports simulation system according to claim 1 wherein said at least one processing stage is configured to use said calculated trajectory to generate updated image data including a simulation of said launched projectile into said sports scene following said calculated trajectory.

3. A sports simulation system according to claim 2 further comprising a projection device coupled to said at least one processing stage, said projection device configured to receive image data from said at least one processing stage and to present said sports scene including said simulation on said display surface.

4. A sports simulation system according to claim 3 wherein said projectile tracking apparatus includes a frame and at least one pair of camera devices mounted on said frame, said camera devices having overlapping fields of view looking across and in front of said display surface and capturing images of said projectile tracking region.

5. A sports simulation system according to claim 4 wherein said camera devices have generally perpendicular fields of view looking across and in front of said display surface from adjacent opposite corners of said frame.

6. A sports simulation system according to claim 3 wherein each camera device is configured to examine captured images to detect pixel clusters resembling a projectile characteristic signature thereby to detect said projectile in said captured images.

7. A sports simulation system according to claim 1 further including an audio system configured to broadcast audio.

8. A sports simulator system according to claim 1 wherein said projectile spin sensing unit comprises at least one imaging device having a field of view aimed downwardly and extending from said projectile launch area toward said display surface, the field of view encompassing approximately a three foot by six foot area.

9. A sports simulation system according to claim 8 wherein said projectile spin sensing unit further comprises at least one illuminator configured to illuminate the field of view of said at least one imaging device at intervals.

10. A sports simulation system according to claim 9 wherein said at least one illuminator comprises an array of light sources.

11. A sports simulation system according to claim 10 wherein said light sources are light emitting diodes.

12. A sports simulation system according to claim 10 comprising a plurality of illuminators at spaced locations with said field of view.

13. A sport simulation system according to claim 1 further comprising a launch area sensing unit configured to capture images of said projectile launch area.

14. A sports simulation system according to claim 13 wherein said launch area sensing unit comprises at least one imaging device having a downwardly aimed field of view encompassing said projectile launch area.

15. A sports simulation system according to claim 14 wherein said launch area sensing unit further comprises at least one illuminator configured to illuminate said projectile launch area.

16. A sports simulation system according to claim 13 wherein said at least one processing stage is configured to process image data received from the launch area sensing unit to determine the angle at which impact with the projectile is made.

17. A sports simulation system comprising:

a projectile tracking apparatus including a frame encompassing a display surface on which a video sequence portraying a sports scene is presented; and

at least one pair of digital camera devices mounted on said frame and having fields of view looking across and in front of said display surface that overlap in a generally perpendicular fashion and encompassing a projectile tracking region, each of said digital camera devices including a first processor configured to process image data and to generate two-dimensional projectile coordinates when a launched, spinning projectile travels from a projectile launch area and through said projectile tracking region towards said display surface and is captured in images acquired by said digital camera devices;

a projectile spin sensing unit positioned intermediate said projectile launch area and said display surface and configured to capture images of a region adjacent said projectile launch area that at least partially overlaps with said projectile tracking region, each captured image comprising a projectile trail that comprises spaced apart high intensity regions resulting from an illuminated discrete elongate reflective marking on said projectile and representing a travel path of said projectile when a projectile is present in said region during image capture; and

a host processor configured to communicate with said digital camera devices and said projectile spin sensing unit, said host processor configured to calculate a three-dimensional trajectory of said projectile taking into account projectile spin rate and tilt using the two-dimensional projectile coordinates received from each first processor and image data output of said projectile spin sensing unit and to output image data including said calculated three-dimensional trajectory, wherein during processing of image data from said projectile spin sensing unit, said host processor is configured to generate a profile of the projectile trail over a plurality of images, determine the projectile trail length per image of said plurality of images, threshold each image to identify the high intensity regions corresponding to the elongate marking on said projectile, determine the distance between the high intensity regions in each image, calculate the spin rate of the projectile using the determined distance, the projectile trail length and the projectile spin sensing unit image frame rate and estimate a spin tilt axis of the projectile using the orientation of the high intensity regions in each image and the relative angle between the longitudinal axis of the high intensity regions and the longitudinal axis of the projectile trail; and

a display unit configured to receive the image data output by said host processor and to present said video sequence including a simulation of said calculated trajectory on said display surface.

18. A sports simulator system according to claim 17 wherein said projectile spin sensing unit comprises at least one imaging device having a field of view aimed downwardly and extending from said projectile launch area toward said display surface.

19. A sports simulation system according to claim 18 wherein said projectile spin sensing unit further comprises at least one illuminator configured to illuminate the field of view of said at least one imaging device at intervals.

20. A sports simulation system according to claim 19 wherein said at least one illuminator comprises an array of light sources.

21. A sports simulation system according to claim 20 wherein said light sources are light emitting diodes.

22. A sports simulation system according to claim 20 comprising a plurality of illuminators at spaced locations with said field of view.

23. A golf simulator comprising:

at least one pair of imaging devices having overlapping fields of view looking across and in front of a display surface from different vantages and configured to capture images of a launched, spinning golf ball travelling from a launch area spaced in front of said display surface into contact with said display surface;

a spin sensing unit configured to capture images of a region in front of said display surface adjacent said launch area, each captured image comprising a golf ball trail that comprises spaced apart high intensity regions resulting from an illuminated discrete elongate reflective marking on a portion of said launched, spinning golf ball and representing a travel path of said launched, spinning golf ball; and

at least one processing stage configured to process image data from the imaging devices and from said spin sensing unit and to determine the three-dimensional positions, velocity, acceleration and spin rate and tilt of the launched, spinning golf ball traveling through said overlapping fields of view, the three-dimensional positions, velocity, acceleration and spin rate and tilt being used by said at least one processing stage to calculate a trajectory of said launched, spinning golf ball into a sports scene projected onto said display surface, wherein during processing of image data from said spin sensing unit, said at least one processing stage is configured to generate a profile of the golf ball trail over a plurality of images, determine the golf ball trail length per image of said plurality of images, threshold each image to identify the high intensity regions corresponding to the elongate marking on said launched, spinning golf ball, determine the distance between the high intensity regions in each image, calculate the spin rate and tilt of the launched, spinning golf ball using the determined distance, the golf ball trail length and the spin sensing unit image frame rate and estimate a spin tilt axis of the golf ball using the orientation of the high intensity regions in each image and the relative angle between the longitudinal axis of the high intensity regions and the longitudinal axis of the golf ball trail; and

a projection unit configured to present said three-dimensional sport scene on said display surface including a simulation of said launched, spinning golf ball following said calculated trajectory.

24. A golf simulator according to claim 23 wherein said spin sensing unit comprises at least one imaging device having a field of view aimed downwardly and extending from a golf ball launch area toward said display surface.

25. A golf simulator according to claim 24 wherein said spin sensing unit further comprises at least one illuminator to illuminate the field of view of said at least one imaging device at intervals.

26. A golf simulator according to claim 25 wherein said at least one illuminator comprises an array of light sources.

27. A golf simulator according to claim 26 wherein said light sources are light emitting diodes.

28. A golf simulator according to claim 26 comprising a plurality of illuminators at spaced locations with said field of view.