Trajectory detection and feedback system for tennis

• US 9,694,238 B2
• Filed: 01/18/2013
• Issued: 07/04/2017
• Est. Priority Date: 09/12/2001
• Status: Expired due to Term

• Current Assignee: Pillar Vision Incorporated
• Original Assignee: Pillar Vision Incorporated
• Inventors: Marty, Alan W., Edwards, Thomas A.
• Primary Examiner(s): D'Agostino, Paul A
• Application Number:
  US13/745,429
• Publication Number:
  US 20130130845A1
• Time in Patent Office: 1,628 Days
• Field of Search: None
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2225/20 with means for remote communication, e.g. internet or the like
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 24/0084 Exercising apparatus with means for competitions, e.g. virtual races
• CPC Class Codes: A63B 55/00 Bags for golf clubs; Stands for golf clubs for use on the course; Wheeled carriers specially adapted for golf bags
• CPC Class Codes: A63B 69/0071 for basketball
• CPC Class Codes: A63B 69/3614 using electro-magnetic, magnetic or ultrasonic radiation emitted, reflected or interrupted by the golf club
• CPC Class Codes: A63B 69/38 for tennis A63B61/006, A63B69/0073 and A63B69/0097 take precedence
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A system for capturing and analyzing a trajectory of a tennis ball or other object associated with a play of a game of tennis and providing feedback is described. The system may be designed to capture and analyze a trajectory of a tennis ball during various activities related to the play of a game of tennis. The system may be configured to provide immediate feedback that may be utilized by a player to improve their performance as well as provide entertainment value above and beyond what is normally associated with the play of a game of tennis. Further, the system may be operable for use in an area where tennis is normally played during normal playing activities, such as player playing a game or practicing on an outdoor tennis court. The system may be operable to account for factors associated with its ambient environment, such as wind, temperature and humidity.

Claims

1. A device for analyzing trajectories of tennis balls, the device comprising:

one or more cameras for recording video frame data used to characterize a trajectory of a tennis ball generated by a player during a tennis related activity;

a logic device designed or configured to i) receive the video frame data, ii) identify the tennis ball in the video frame data, iii) determine when a tennis racquet strikes the tennis ball, iv) generate trajectory parameters that characterize one or more states of the tennis ball along its trajectory based on the identified tennis ball in the video frame data, including at least one trajectory parameter measured by the logic device that characterizes a state of the tennis ball when the tennis racquet is determined by the logic device to strike the tennis ball, and v) generate feedback information using the trajectory parameters, wherein the feedback information indicates the at least one trajectory parameter, wherein the logic device is designed or configured to identify a net associated with a tennis court including an upper edge of said net in the video frame data, wherein the logic device is further designed or configured to measure a height of the tennis ball relative to the upper edge of said net when the tennis ball is above the net during a shot of the tennis ball, and wherein the feedback information indicates the measured height of the tennis ball; and

at least one output mechanism for providing the feedback information.

2. The device of claim 1, wherein the tennis related activity includes a toss of the tennis ball for a serve and wherein the logic device is further designed or configured to measure the trajectory of the tennis ball during the toss and prior to impact with a racquet.

3. The device of claim 1, wherein the feedback information includes one or more of the following 1) a height of a serve toss, 2) a lean of the serve toss, 3) a spin of the serve toss, 4) a consistency of a plurality of serve tosses, 5) an impact position height of a serve, 6) an impact position lean of the serve, 7) a consistency of an impact position of a plurality of serves, 8) an initial speed of the serve, 9) an initial angle of the serve, 10) an initial direction of the serve, 11) an initial spin of the serve, 12) a consistency of the initial speed of the plurality of serves, 13) a consistency of the angle of serve of the plurality of serves, 14) a consistency of the direction of the plurality of serves, 15) a consistency of the spin of the plurality of serves, 16) a calculated speed of the serve, 17) a consistency of the calculated speed of the plurality of serves, 18) a calculated landing speed of the serve, 19) a location of the serve at landing, 20) a spin of the serve at landing, 21) a direction vector of the serve at landing, 22) a consistency of the calculated landing speed for the plurality of serves, 23) a consistency of the location at landing of the plurality of server, 24) a consistency of the spin at landing of the plurality of serves, 25) a consistency of the direction vector of the plurality of serves, 26) a measured landing speed of the serve, 27) a measured location of the serve, 28) a measured spin of the serve, 29) a measured direction vector of the serve, 30) a consistency of the measured landing speed of the plurality of serves, 31) a consistency of the measured location of the plurality of serves, 32) a consistency of the spin measured for the plurality of serves, 33) a consistency of the direction vector measured for the plurality of serves, 34) a location of the crossing point of the net of the tennis ball, 35) a release location of the serve toss and 36) combinations thereof.

4. The device of claim 1, wherein the logic device is further designed or configured to identify a position of a body element of the player participating in the tennis related activity in the video frame data and to determine the position of the body element relative to a marking on the court.

5. The device of claim 1, wherein the logic device is further designed or configured to determine whether a tennis ball hit by a player on a tennis court is inside of or outside of one or more boundary lines associated with the tennis court.

6. The device of claim 1, wherein the logic device is further designed or configured to identify a racquet in the video frame data and wherein the logic device is further designed or configured to determine a position of the racquet as a function of time, a velocity of the racquet as a function of time, an orientation of the racquet as a function of time or combinations thereof.

7. The device of claim 1, wherein the logic device is further designed or configured to identify a body element of the player participating in the tennis related activity in the video frame data and wherein the logic device is further designed or configured to determine a position of the body element, an orientation of the body element, a velocity of the body element or combinations thereof, as a function of time and generate feedback information related to one or more of the position of the body element, the orientation of the body element or the velocity of the body element.

8. The device of claim 1, wherein the logic device is further designed or configured to determine, for the purposes of calibration, a distance from the device to one or more of the tennis ball, a racquet, a marking on a tennis court, a net on the tennis court, a vertical surface against which the tennis ball is being hit or a player hitting the tennis ball.

9. The device of claim 1, further comprising one or more sensors for determining an orientation of the device and wherein the one or more sensors comprise accelerometers or tilt sensors.

10. The device of claim 1, wherein the output mechanism includes a wireless interface for outputting the feedback information to one or more remote devices.

11. The device of claim 10, wherein the remote device is worn.

12. The device of claim 1, further comprising:

a housing for the one or more cameras, the logic device, and the at least one output mechanism, the housing having a weight and form factor which facilitate one or more of transport, storage, unobtrusive set-up, calibration, or operation of the device.

13. The device of claim 1, further comprising an input mechanism.

14. The device of claim 13, wherein the input mechanism is a touch screen display.

15. The device of claim 13, wherein the input mechanism is a wireless interface for receiving input from a remote device.

16. The device of claim 1, wherein the logic device is a general purpose computer comprising:

a processor, a data storage device, RAM, operating system software, device interfaces, device drivers and trajectory analysis software.

17. The device of claim 1, wherein the device is capable of one of autonomous set-up, autonomous calibration, autonomous operation or combinations thereof.

18. The device of claim 1, wherein after manual input of data by a user, a confirmation of data determined by the device, the logic device is further designed or configured to complete a calibration procedure.

19. The device of claim 1, further comprising:

a memory storage device for storing trajectory session information wherein the trajectory session information comprises one or more of

     1) digitized video frame data, trajectory information and feedback information generated for a plurality of trajectories,

     2) a trajectory session time,

     3) a trajectory session date,

     4) a trajectory session location and combinations thereof.

20. The device of claim 1, wherein the device is designed to determine for a plurality of related trajectories captured by the device a consistency for at least one of the trajectory parameters generated for each of the plurality of related trajectories.

21. The device of claim 20, wherein the consistency is determined by calculating a statistical deviation.

22. The device of claim 1, wherein the logic device is further designed or configured to store data related to one or the trajectory of the tennis ball, movements of the player, movements of a racquet captured in the video frame data for use in a video simulation related to tennis.

23. The device of claim 1, wherein the logic device is further designed or configured to identify a boundary line of a tennis court in the video frame data and to determine whether the tennis ball is inside or outside of the boundary line based on the identified tennis ball and the identified boundary line in the video frame data, and wherein the feedback information indicates whether the tennis ball is inside or outside of the boundary line.

24. The device of claim 1, wherein the at least one trajectory parameter includes a height at which the tennis racquet strikes the tennis ball.

25. The device of claim 1, wherein the logic device is designed or configured to determine a height of a serve toss based on the identified tennis ball in the video frame data, and wherein the feedback information indicates the height of the serve toss.

26. The device of claim 1, wherein the logic device is designed or configured to measure a spin of the tennis ball during a serve toss based on the identified tennis ball in the video frame data, and wherein the feedback information indicates the spin.

27. The device of claim 23, wherein the at least one trajectory parameter includes a height at which the tennis racquet strikes the tennis ball.

28. The device of claim 27, wherein the logic device is designed or configured to determine a height of a serve toss based on the identified tennis ball in the video frame data, and wherein the feedback information indicates the height of the serve toss.

38. The device of claim 1, wherein the logic device is configured to calculate an average height of at least one tennis ball above the upper edge of said net for multiple tennis shots, and wherein the feedback information indicates the average height.

39. The device of claim 24, wherein the logic device is configured to calculate an average height at which the tennis racquet strikes a tennis ball for multiple tennis shots, and wherein the feedback information indicates the average height.

29. A device for analyzing trajectories of tennis balls, the device comprising:

one or more cameras for recording video frame data used to characterize a trajectory of a tennis ball generated by a player during a tennis related activity;

a logic device designed or configured to i) receive the video frame data, ii) identify the tennis ball in the video frame data, iii) determine when a tennis racquet strikes the tennis ball, iv) generate trajectory parameters that characterize one or more states of the tennis ball along its trajectory based on the identified tennis ball in the video frame data, including at least one trajectory parameter measured by the logic device that characterizes a state of the tennis ball when the tennis racquet is determined by the logic device to strike the tennis ball, and v) generate feedback information using the trajectory parameters, wherein the feedback information indicates the at least one trajectory parameter, wherein the at least one trajectory parameter includes a height at which the tennis racquet strikes the tennis ball, wherein the logic device is further designed or configured to identify a boundary line of a tennis court in the video frame data and to determine whether the tennis ball is inside or outside of the boundary line based on the identified tennis ball and the identified boundary line in the video frame data, wherein the feedback information indicates whether the tennis ball is inside or outside of the boundary line, wherein the logic device is designed or configured to determine a height of a serve toss based on the identified tennis ball in the video frame data, and wherein the feedback information indicates the height of the serve toss, wherein the logic device is further designed or configured to identify a net associated with a tennis court including an upper edge of said net in the video frame data, wherein the logic device is further designed or configured to determine a position of the tennis ball relative to the upper edge of said net when the tennis ball is above the net, and wherein the feedback information indicates the position of the tennis ball relative to the upper edge of said net; and

at least one output mechanism for providing the feedback information.

30. The device of claim 29, wherein the logic device is designed or configured to measure a spin of the tennis ball during the serve toss based on the identified tennis ball in the video frame data, and wherein the feedback information indicates the spin.

31. A device for analyzing trajectories of tennis balls, the device comprising:

one or more cameras for recording video frame data used to characterize a trajectory of a tennis ball generated by a player during a tennis related activity;

a logic device designed or configured to i) receive the video frame data, ii) identify the tennis ball in the video frame data, iii) generate trajectory parameters that characterize one or more states of the tennis ball along its trajectory based on the identified tennis ball in the video frame data, and iv) generate feedback information using the trajectory parameters, wherein the feedback information indicates at least one of the one or more states, wherein the logic device is further designed or configured to identify a net associated with a tennis court including an upper edge of said net in the video frame data, wherein the logic device is further designed or configured to measure a height of the tennis ball relative to the upper edge of said net when the tennis ball is above the net during a shot of the tennis ball, and wherein the feedback information indicates the measured height of the tennis ball relative to the upper edge of said net; and

at least one output mechanism for providing the feedback information.

32. A method for analyzing trajectories of tennis balls, comprising:

recording video frame data by one or more cameras, the video frame data including images of a tennis ball during a tennis related activity at a tennis court;

receiving the video frame data at a logic device;

identifying, by the logic device, the tennis ball in the video frame data;

generating, by the logic device, trajectory parameters that characterize one or more states of the tennis ball along its trajectory;

generating, by the logic device, feedback information using the trajectory parameters;

identifying, by the logic device, a net associated with the tennis court in the video frame data;

measuring, by the logic device, a height of the tennis ball relative to an upper edge of said net when the tennis ball is above the net during a shot of the tennis ball, wherein the feedback information indicates the measured height; and

outputting the feedback information by an output mechanism.

33. The method of claim 32, further comprising determining, by the logic device, when a tennis racquet strikes the tennis ball, wherein the trajectory parameters includes at least one trajectory parameter that characterizes a state of the tennis ball when the tennis racquet is determined to strike the tennis ball, and wherein the feedback information indicates the at least one trajectory parameter.

34. The method of claim 33, wherein the at least one trajectory parameter includes a height at which the tennis racquet strikes the tennis ball.

35. The method of claim 34, further comprising measuring, by the logic device, a height of a serve toss based on the identified tennis ball in the video frame data, wherein the feedback information indicates the measured height of the serve toss.

36. The method of claim 35, further comprising measuring, by the logic device, a spin of the tennis ball during the serve toss based on the identified tennis ball in the video frame data, wherein the feedback information indicates the measured spin.

37. A device for analyzing trajectories of tennis balls, the device comprising:

one or more cameras for recording video frame data used to characterize a trajectory of a tennis ball generated by a player during a tennis related activity;

a logic device designed or configured to i) receive the video frame data, ii) identify the tennis ball in the video frame data, iii) determine when a tennis racquet strikes the tennis ball, iv) generate trajectory parameters that characterize one or more states of the tennis ball along its trajectory based on the identified tennis ball in the video frame data, including at least one trajectory parameter measured by the logic device that characterizes a state of the tennis ball when the tennis racquet is determined by the logic device to strike the tennis ball, and v) generate feedback information using the trajectory parameters, wherein the feedback information indicates the at least one trajectory parameter, wherein the logic device is further designed or configured to identify a net associated with a tennis court, wherein the logic device is further designed or configured to measure a height of the tennis ball when the tennis ball is above the net, and wherein the feedback information indicates the measured height; and

at least one output mechanism for providing the feedback information.