TRACKING AN OBJECT WITH MULTIPLE ASYNCHRONOUS CAMERAS

• US 20080219509A1
• Filed: 03/19/2007
• Published: 09/11/2008
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: White, Marvin S., Alt, Alina
• Granted Patent: US 8,335,345 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: A63B 2024/0028 Tracking the path of an object, e.g. a ball inside a soccer pitch
• CPC Class Codes: A63B 2024/0034 during flight
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 2102/18 Baseball, rounders or similar games
• CPC Class Codes: A63B 2102/32 Golf
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/20 Distances or displacements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/62 Time or time measurement used for time reference, time stamp, master time or clock signal
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2220/89 Field sensors, e.g. radar systems
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2243/0066 Rugby; American football
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 69/3658 Means associated with the ball for indicating or measuring, e.g. speed, direction
• CPC Class Codes: A63B 71/06 Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 : Multi-camera tracking

Abstract

The path and/or position of an object is tracked using two or more cameras which run asynchronously so there is need to provide a common timing signal to each camera. Captured images are analyzed to detect a position of the object in the image. Equations of motion for the object are then solved based on the detected positions and a transformation which relates the detected positions to a desired coordinate system in which the path is to be described. The position of an object can also be determined from a position which meets a distance metric relative to lines of position from three or more images. The images can be enhanced to depict the path and/or position of the object as a graphical element. Further, statistics such as maximum object speed and distance traveled can be obtained. Applications include tracking the position of a game object at a sports event.

Claims

1. A method for determining a path of a moving object, comprising:

receiving images of the moving object from a plurality of cameras at different time points during a time interval, the plurality of cameras capture the images asynchronously;

determining a position of the moving object in each image; and

solving for coefficients of equations of motion of the moving object, based on the determined positions, the equations of motion with the solved coefficients describe a path of the moving object during the time interval.

2. The method of claim 1, wherein:

each determined position represents a point on a line of position from the camera to the object.

3. The method of claim 1, wherein:

the solving comprises solving for a matrix which includes the coefficients of the equations of motion, based on matrices which include the determined positions and matrices which include transform coefficients, the transform coefficients associate one or more coordinate systems of the images with a coordinate system in which the path is described.

4. The method of claim 1, wherein:

the solving comprises satisfying a least square error criteria for errors between lines of position of the moving object which are based on the determined positions and positions of the moving object which are based on the equations of motion.

5. The method of claim 1, wherein:

the equations of motion with the solved coefficients describe the path of the moving object in three orthogonal directions at any time during the time interval.

6. The method of claim 1, wherein:

the coefficients describe a position, velocity and acceleration of the moving object along the path of the moving object at any time during the time interval.

7. The method of claim 1, further comprising:

identifying the different time points at which the images are received using a common clock signal which is independent of the cameras.

8. The method of claim 1, wherein:

the equations of motion are linear equations.

9. The method of claim 1, wherein:

the equations of motion include at least one non-linear equation.

10. The method of claim 1, wherein

the images are from video of the moving object, and the method further comprises enhancing the video to depict at least a portion of the path of the moving object.

11. The method of claim 1, further comprising:

providing statistics regarding the path of the moving object.

12. The method of claim 1, further comprising:

reporting based on the solving.

13. The method of claim 1, wherein:

the cameras sense visible light.

14. The method of claim 1, wherein:

the cameras sense electromagnetic radiation outside a range of visible light.

15. A system for determining a path of a moving object used at an event, comprising:

a plurality of cameras capturing images of the moving object at the event during a time interval, the plurality of cameras capture the images asynchronously; and

at least one processing facility which receives the captured images from the plurality of cameras, the at least one processing facility;

a) determines a position of the moving object in each image, and b) solves for coefficients of equations of motion of the moving object, based on the determined positions, the equations of motion with the solved coefficients describe a path of the moving object during the time interval.

16. The system of claim 15, wherein:

each determined position represents a point on a line of position from the camera to the object.

17. The system of claim 15, wherein:

the at least one processing facility solves for the coefficients by solving for a matrix which includes the coefficients of the equations of motion, based on matrices which include the determined positions and matrices which include transform coefficients, the transform coefficients associate one or more coordinate systems of the images with a coordinate system in which the path is described.

18. The system of claim 15, wherein:

the at least one processing facility solves for the coefficients in accordance with a least square error criteria for errors between lines of position of the moving object which are based on the determined positions and positions of the moving object which are based on the equations of motion.

19. The system of claim 15, wherein:

the equations of motion with the solved coefficients describe the path of the moving object in three orthogonal directions at any time during the time interval.

20. The system of claim 15, wherein:

the coefficients describe a position, velocity and acceleration of the moving object along the path of the moving object at any time during the time interval.

21. The system of claim 15, wherein:

the at least one processing facility identifies different time points at which the captured images are received from the plurality of cameras using a common clock signal which is independent of the plurality of cameras, and the at least one processing facility solves for the coefficients for each of the different time points.

22. The system of claim 15, wherein:

the captured images are from video of the moving object, and the at least one processing facility enhances the video to depict at least a portion of the path of the moving object.

23. The system of claim 15, wherein:

the at least one processing facility provides statistics regarding the path of the moving object.

24. The system of claim 15, wherein:

the at least one processing facility reports based on the described path.

25. The system of claim 15, wherein:

the cameras sense visible light.

26. The system of claim 15, wherein:

the cameras sense electromagnetic radiation outside a range of visible light.

27. At least one processor readable storage device having processor readable code embodied thereon for programming at least one processor to perform a method, the method comprising:

receiving images of the moving object from a plurality of cameras at different time points during a time interval, the plurality of cameras capture the images asynchronously;

determining a position of the moving object in each image; and

solving for coefficients of equations of motion of the moving object, based on the determined positions, the equations of motion with the solved coefficients describe a path of the moving object during the time interval.

28. The at least one processor readable storage device of claim 27, wherein:

each determined position represents a point on a line of position from the camera to the object.

29. The at least one processor readable storage device of claim 27, wherein:

the solving comprises solving for a matrix which includes the coefficients of the equations of motion, based on matrices which include the determined positions and matrices which include transform coefficients, the transform coefficients associate one or more coordinate systems of the images with a coordinate system in which the path is described.

30. The at least one processor readable storage device of claim 27, wherein:

the solving comprises satisfying a least square error criteria for errors between lines of position of the moving object which are based on the determined positions and positions of the moving object which are based on the equations of motion.

31. The at least one processor readable storage device of claim 27, wherein:

the equations of motion with the solved coefficients describe the path of the moving object in three orthogonal directions at any time during the time interval.

32. The at least one processor readable storage device of claim 27, wherein:

the coefficients describe a position, velocity and acceleration of the moving object along the path of the moving object at any time during the time interval.

33. The at least one processor readable storage device of claim 27, further comprising:

identifying the different time points at which the images are received using a common clock signal which is independent of the cameras.

34. The at least one processor readable storage device of claim 27, wherein:

the images are from video of the moving object, and the method performed further comprises enhancing the video to depict at least a portion of the path of the moving object.

35. The at least one processor readable storage device of claim 27, wherein:

the method performed further comprises providing statistics regarding the path of the moving object.

36. The at least one processor readable storage device of claim 27, wherein:

the method performed further comprises reporting based on the solving.

37. A method for determining a position of a moving object, comprising:

receiving at least first, second and third images of the moving object from a plurality of cameras at different time points during a time interval, the plurality of cameras capture the images asynchronously;

determining a position of the moving object in the at least first, second and third images;

determining lines of position based on the determined positions in the at least first, second and third images; and

determining a first position which satisfies a metric based on distances from the first position to the lines of position.

38. The method of claim 37, wherein:

the metric is based on minimizing a sum of the distances.

39. The method of claim 37, wherein:

the metric is based on minimizing a sum of squares of the distances.

40. The method of claim 37, wherein:

the first position is a three-dimensional position.

41. The method of claim 37, further comprising:

enhancing at least one of the images to depict the first position.

42. The method of claim 37, further comprising:

reporting the determined first position.

43. A system for determining a position of a moving object used at an event, comprising:

a plurality of cameras capturing at least first, second and third images of the moving object from a plurality of cameras at different time points during a time interval, the plurality of cameras capture the images asynchronously; and

at least one processing facility which receives the at least first, second and third images from the plurality of cameras, the at least one processing facility;

a) determines a position of the moving object in the at least first, second and third images, b) determines lines of position based on the determined positions in the at least first, second and third images, and c) determines a first position which satisfies a metric based on distances from the first position to the lines of position.

44. The system of claim 43, wherein:

the metric is based on minimizing a sum of the distances.

45. The system of claim 43, wherein:

the metric is based on minimizing a sum of squares of the distances.

46. The system of claim 43, wherein:

the first position is a three-dimensional position.

47. The system of claim 43, wherein:

the at least one processing facility reports the determined first position.

48. The system of claim 43, wherein:

the at least one processing facility enhances at least one of the images to depict the first position.

49. At least one processor readable storage device having processor readable code embodied thereon for programming at least one processor to perform a method, the method comprising:

receiving at least first, second and third images of the moving object from a plurality of cameras at different time points during a time interval, the plurality of cameras capture the images asynchronously;

determining a position of the moving object in the at least first, second and third images;

determining lines of position based on the determined positions in the at least first, second and third images; and

determining a first position which satisfies a metric based on distances from the first position to the lines of position.

50. The at least one processor readable storage device of claim 49, wherein:

the metric is based on minimizing a sum of the distances.

51. The at least one processor readable storage device of claim 49, wherein:

the metric is based on minimizing a sum of squares of the distances.

52. The at least one processor readable storage device of claim 49, wherein:

the first position is a three-dimensional position.

53. The at least one processor readable storage device of claim 49, wherein:

the at least one processing facility enhances at least one of the images to depict the first position.

54. The at least one processor readable storage device of claim 49, wherein:

the at least one processing facility reports the determined first position.