Method for representing real-time motion

• US 6,707,487 B1
• Filed: 02/22/2000
• Issued: 03/16/2004
• Est. Priority Date: 11/20/1998
• Status: Expired due to Term

• Current Assignee: Maxx Holdings, Inc.
• Original Assignee: In the Play, Inc.
• Inventors: Aman, James A., Haller, William R.
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US09/510,922
• Time in Patent Office: 1,484 Days
• Field of Search: H04/N.5225, 701/300, 364/156, 348/169, 348/162, 348/164, 348/208.14, 348/77, 250/203, 382/103, 702/150, 702/151, 702/152
• US Class Current: 348/169
• CPC Class Codes: A63B 2024/0025 Tracking the path or location of one or more users, e.g. players of a game
• CPC Class Codes: A63B 2102/24 Ice hockey
• CPC Class Codes: A63B 2208/12 specially adapted for children
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/13 Relative positions
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2220/807 Photo cameras
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 71/0605 Decision makers and devices using detection means facilitating arbitration
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A system 100 for tracking the movement of multiple objects within a predefined area using a continuation of overhead X-Y tracking cameras 24 with attached frequency selective filter 24f. Also employed are perspective Z filming cameras sets 30. Objects to be tracked, such as player 17, have been marked to include some form of frequency selective reflective material such as an ink. Typical markers include patches 7r and 7l, sticker 9 and tape 4a as well as additional body joint markers 17af through 17l. System 100 radiates selected energy 23a throughout the predefined area of tracking that is specifically chosen to reflect off said reflective materials used to mark for instance player 17. The reflected energy is then received by tracking cameras 24 while all other ambient light is blocked by filter 24f. Local Computer System 60 continuously captures images from said tracking cameras 24 which include only the minimum information created by said reflected energy. System 60 efficiently locates said markings on said multiple objects and uses this location information to determine for each marking its angle of rotation, angle of azimuth and distance from a designated origin 17o local to player 17. Local origin 17o is then expressed as a three-dimensional coordinate with respect to the origin of the playing venue 2a. The continuous stream of tracked three-dimensional coordinates, defining the body joints on players such as 17, is then transmitted to a remote computer where it can be used to drive a graphic re-animation of the object movement. Along with this re-animation, additional performance measurements may be derived from the continuous stream and automatically made available in real-time.

Claims

1. A live performance viewing and analysis system comprising:

a tracking system for continuously tracking the location of multiple body points on one and more players in a defined physical space; and

a computer operatively coupled to the tracking system for updating in real time the virtual location of all tracked body points on each one and more players in a virtual space corresponding to the physical location of all tracked body points on each one and more players in the physical space, for updating a view of the virtual space, and for providing at least one indicium of performance of the one and more players moving in the physical space, wherein the at least one indicium is or is derived from a measure of a movement parameter of the one and more players.

2. The live performance viewing and analysis system of claim 1 wherein the performance is digitally compressed into a first coordinate system of body point locations expressed relative to a corresponding player'"'"'s origin and then digitally compressed into a second coordinate system of player origins expressed relative to the corresponding physical space origin.

3. The live performance viewing and analysis system of claim 2 wherein the view presented through the operatively coupled computer of the one and more players can be controllably changed to match any desired perspective based upon the digitally compressed first and second coordinate systems.

4. The live performance viewing and analysis system of claim 1 wherein the tracking system further tracks the location and movement of player equipment and the contest object such as a puck or ball.

5. The live performance viewing and analysis system of claim 4 further providing at least one indicium indicative of the movement of equipment and or the contest object.

6. The live performance viewing and analysis system of claim 1 wherein the tracking system further comprises an automated system for following the movement of one and more players within a predefined area using an energy source that emits energy at least including a narrow band of tracking energy, the system comprising:

one and more marks placed upon the one and more players that reflect only the narrow band of tracking energy;

one and more tracking detectors placed throughout the predefined area that receive the reflected narrow band of tracking energy; and

a computer system responsive to the detectors that analyzes the narrow band of tracking energy received by the detectors and creates a database of movement information related to the one and more marks placed upon the one and more players.

7. The automated system of claim 6 wherein the narrow band of tracking energy is non-visible.

8. The automated system of claim 7 further comprising one and more lamps emitting the narrow band of non-visible tracking energy.

9. The automated system of claim 8 wherein the one and more lamps are physically separated from the one and more detectors.

10. The automated system of claim 6 wherein the narrow band of tracking energy is constantly radiated, reflected, and received without interruption.

11. The automated system of claim 6 wherein the one and more marks are embedded onto the surface of the one and more players.

12. The automated system of claim 11 wherein the one and more marks are optionally embedded onto the surface of a material that is attached to the one and more players in such a way that it remains flat to the surface of the object.

13. The automated system of claim 6 wherein one or more of the marks placed upon each object bears indicia uniquely identifying that object.

14. The automated system of claim 13 wherein the computer system determines the identity of each object based solely upon the information obtained from the one or more indicia bearing marks.

15. The automated system of claim 13 wherein the detectors are one and more first tracking cameras arranged in a grid formation overhead of the predefined area where the one and more players move, the first tracking cameras being used to track the X-Y movement of the one and more players.

16. The automated system of claim 15 wherein at least one of the one or more indicia bearing marks are placed upon a top surface of each of the one and more players such that it is consistently in the view of the overhead first tracking cameras.

17. The automated system of claim 16 wherein the computer system determines the identity of each object based solely upon the information obtained from the at least one indicia bearing mark that was placed upon the top of each one and more object as viewed through the grid of overhead first tracking X-Y cameras.

18. The automated system of claim 6 wherein the detectors are one and more first tracking cameras arranged in a grid formation overhead of the predefined area where the one and more players move, the first tracking cameras being used to track the X-Y movement of the one and more players.

19. The automated system of claim 18 wherein additional detectors in the form of one and more second tracking cameras are placed at perspective views to the movement of the one and more players.

20. The automated system of claim 19 wherein additional detectors in the form of one and more filming cameras are included in order to receive visible light images corresponding to the non-visible light images received by the first and second tracking cameras.

21. The automated system of claim 19 wherein the computer determines Z dimension information about the one and more marks placed upon each of the one and more players based upon images from the one and more second tracking cameras.

22. The automated system of claim 21 wherein the one and more first and second tracking cameras are further fitted with a lens filter to block visible light.

23. The automated system of claim 20 wherein the computer system receives the visible light images from the filming cameras and stores these images cross-indexed with the tracking information obtained from the first and second tracking cameras.

24. The automated system of claim 23 wherein the location information determined by the computer system using the images of the first and second tracking cameras and pertaining to any and all marks is used to selectively process a portion of the images of the filming cameras by performing a comparison of these images to the pre-known background thereby identifying a foreground object without having to process all the pixels of the images.

26. The automated system of claim 5 wherein the shape of the one and more marks provides information about the object'"'"'s orientation.

27. The automated system of claim 26 wherein the spatial arrangement of the one and more marks provides information about the object'"'"'s orientation.

28. The automated system of claim 27 wherein the information calculated by the computer system concerning the location of the one and more marks and therefore of the location of the one and more players onto which the marks have been placed comprises the current location, direction of movement, velocity, acceleration, and orientation of the object associated with the time the images were captured from which the information was determined.

29. The automated system of claim 14 wherein the information calculated by the computer system is automatically stored in a database cross-linked to and retrievable by the identified object.

30. The automated system of claim 29 wherein the information calculated and automatically stored in the database by the computer system is further analyzed to create new comparison information that is also stored in the database.

31. The automated system of claim 30 further comprising a remote computer system and wherein the information calculate and automatically stored in the database by the computer system as well as the new comparison information is transmittable to the remote computer system.

32. The automated system of claim 31 wherein the remote computer system is capable of recreating a graphic representation of the movement of the tracked players based upon the transmitted information.

25. The automated system of claim wherein the location of each mark is tracked individually.