Smart-court system and method for providing real-time debriefing and training services of sport games

• US 10,391,378 B2
• Filed: 05/16/2018
• Issued: 08/27/2019
• Est. Priority Date: 02/23/2012
• Status: Active Grant

• Current Assignee: PlaySight Interactive, Ltd.
• Original Assignee: PlaySight Interactive, Ltd.
• Inventors: Shachar, Chen, Khazanov, Evgeni, Ben Zur, Yoram
• Primary Examiner(s): D'Agostino, Paul A
• Application Number:
  US15/981,127
• Publication Number:
  US 20180264342A1
• Time in Patent Office: 468 Days
• Field of Search: None
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 71/02 for large-room or outdoor sporting games
• CPC Class Codes: A63B 71/04 for small-room or indoor sporting games
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/42 of sport video content

Abstract

A Smart-court system, adaptive to constrained sport environment, for enabling real time analysis and debriefing of sport activities is provided herein. The Smart-court system is comprised of: (i) an automatic recording system comprising a plurality of video cameras located in a court, arranged to real-time (RT) recording of a sport session and utilizing automatic calibration and stabilization module; and (ii) a data processing system comprising: (a) a capture module for grabbing a video stream; (b) an objects'"'"' detector module arranged to extract during the RT sport session, the objects from the foreground of each frame; (c) an event module for automatically analyzing, the motion and the activities of the tracked objects for automatically identifying and classifying events, creating a synchronized event log and calculating statistics that occurred during the RT sport session; and (d) a presentation module enabling to perform instant debriefing, combined biomechanical and tactical analysis of the video.

Claims

1. A Smart-court system, comprising:

a computer programmed to implement;

a capture module for receiving a video stream of a sport session;

an object detector module, for extracting at least one object from frames of the video stream, for tracking at least one of the at least one extracted object as appearing in the video stream;

an event module for automatically analyzing motion of the tracked at least one object, for automatically identifying a plurality of events and classifying the identified events into types based on said tracking and analyzing, and for creating a synchronized event log of the classified events based on said identifying and classifying; and

a presentation module for allowing a user to selectively play at least one of the identified events classified into the types, by type, based on the synchronized event log.

2. The smart-court system of claim 1, further comprising a tracking module configured to identify a part of the sport session, the part starting with a stroke and ending when a player gains a score.

3. The smart-court system of claim 1, wherein one of the tracked objects is a player and the computer is further programmed to yield a pattern of behavior of the player from more than one of the identified events.

4. The smart-court system of claim 3, wherein the computer is further programmed to generate a profile of the player, the profile comprising information based at least in part, on the yielded pattern of behavior.

5. The smart-court system of claim 1, wherein said computer is further programmed to perform automatic calibration by dividing a court as appearing in the video stream using at least one grid, and in each frame of the video stream, checking the grid for deviation of a line of the court from the grid.

6. The smart-court system of claim 1, wherein the computer is further programmed to identify a start and end of a sport game or training session and to automatically identify a change of status of the sport game or training session.

7. The smart-court system of claim 1, wherein the computer is further programmed to identify a change of location of players when the players switch sides on a court.

8. The smart-court system of claim 1, further comprising an additional techniques and strategies module configured to provide recommendations to a player on how to correct activity in a sport game based on said analyzing.

9. The smart-court system of claim 1, wherein said computer is further programmed to use said identifying of the events, for cutting out idle time in the video stream.

10. The smart-court system of claim 1, wherein said presentation module is further configured to enable playing an abridged video displaying only some of the identified events by jumping between some of the identified events according to a user-input query and the synchronized log event.

11. The smart-court system of claim 1, wherein the presentation module is further configured to play a virtual 3D dynamic scene using a 3D engine which simulates objects behavior based on said analyzing and the synchronized event log.

12. The smart-court system of claim 1, wherein said computer is further configured to identify occurrence of a mental event based on said analyzing.

13. A method for implementing a smart court system, the method comprising:

by a computer, receiving a video stream of a sport session;

extracting at least one of object from frames of the video stream, for tracking at least one of the at least one extracted object as appearing in the video stream;

automatically analyzing motion of the tracked at least one object;

automatically identifying a plurality of events and classifying the identified events into types, based on said tracking and analyzing; and

creating a synchronized event log of the classified events based on said identifying and classifying; and

allowing a user to selectively play at least one of the identified events classified into types, by type, using the synchronized event log.

14. The method of claim 13, further comprising identifying a part of the sport session, the part starting with a stroke and ending when a player gains a score.

15. The method of claim 13, further comprising yielding a pattern of behavior of a player from more than one of the identified events.

16. The method of claim 13, further comprising identifying a change of status of the sport session according to a predefined gesture of a user.

17. The method of claim 13, further comprising creating a customized video by integrating events according to a given criteria determined by a user.

18. The method of claim 13, further comprising identifying unforced error based on identifying timing and position of a player before striking a ball.

19. The method of claim 13, further comprising automatically calculating a game score based on said identified and classified events and on an analysis of player and ball position and motion.

20. The method of claim 13, further comprising providing performance matching between players.