Tracking an object with multiple asynchronous cameras

• US 8,705,799 B2
• Filed: 10/29/2012
• Issued: 04/22/2014
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: White, Marvin S, Alt, Alina
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Brooks, Julian
• Application Number:
  US13/662,782
• Publication Number:
  US 20130058532A1
• Time in Patent Office: 540 Days
• Field of Search: 382103-107, 348/47, 348/48, 348/135, 348/138, 348/139, 348/142, 348/154, 348/159, 348/169, 348/170
• US Class Current: 382/103
• CPC Class Codes: A63B 2024/0028 Tracking the path of an object, e.g. a ball inside a soccer pitch
• CPC Class Codes: A63B 2024/0034 during flight
• CPC Class Codes: A63B 2102/02 Tennis
• CPC Class Codes: A63B 2102/18 Baseball, rounders or similar games
• CPC Class Codes: A63B 2102/32 Golf
• CPC Class Codes: A63B 2220/05 Image processing for measuring physical parameters
• CPC Class Codes: A63B 2220/20 Distances or displacements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/62 Time or time measurement used for time reference, time stamp, master time or clock signal
• CPC Class Codes: A63B 2220/806 Video cameras
• CPC Class Codes: A63B 2220/89 Field sensors, e.g. radar systems
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2243/0066 Rugby; American football
• CPC Class Codes: A63B 24/0003 Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
• CPC Class Codes: A63B 24/0021 Tracking a path or terminating locations
• CPC Class Codes: A63B 69/3658 Means associated with the ball for indicating or measuring, e.g. speed, direction
• CPC Class Codes: A63B 71/06 Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/292 : Multi-camera tracking

Abstract

The path and/or position of an object is tracked using two or more cameras which run asynchronously so there is need to provide a common timing signal to each camera. Captured images are analyzed to detect a position of the object in the image. Equations of motion for the object are then solved based on the detected positions and a transformation which relates the detected positions to a desired coordinate system in which the path is to be described. The position of an object can also be determined from a position which meets a distance metric relative to lines of position from three or more images. The images can be enhanced to depict the path and/or position of the object as a graphical element. Further, statistics such as maximum object speed and distance traveled can be obtained. Applications include tracking the position of a game object at a sports event.

Claims

1. A method for determining a position of a moving object, comprising:

receiving first, second and third images of the moving object, a first camera captures the first and third images, a second camera captures the second image asynchronously to the first camera capturing the first and third images, the first, second and third images are captured at first, second and third time points, respectively;

determining a location of the moving object in the first image in a pixel space of the first camera, and a first vector which extends from an origin of a coordinate system of the first camera to the location of the moving object in the pixel space of the first camera;

determining a location of the moving object in the second image in a pixel space of the second camera, and a second vector which extends from an origin of a coordinate system of the second camera to the location of the moving object in the pixel space of the second camera, the coordinate system of the second camera is different than the coordinate system of the first camera;

determining a location of the moving object in the third image in the pixel space of the first camera, and a third vector which extends from the origin of the coordinate system of the first camera to the location of the moving object in the pixel space of the first camera;

transforming, to a world coordinate system, the first and third vectors from the coordinate system of the first camera, and the second vector from coordinate system of the second camera;

determining an instantaneous position of the moving object in the world coordinate system at a time point which is different than the first, second and third time points, by determining a position in 3-d space which satisfies a metric, the metric is based on a first distance from the position in 3-d space to a first line represented by the first vector, a second distance from the position in 3-d space to a second line represented by the second vector, and a third distance from the position in 3-d space to a third line represented by the third vector; and

enhancing a video signal to depict the instantaneous position of the moving object.

2. The method of claim 1, wherein:

the metric is based on minimizing a sum of the first, second and third distances.

3. The method of claim 1, wherein:

the metric is based on minimizing a sum of squares of the first, second and third distances.

4. The method of claim 1, wherein:

the determining the position in 3-d space comprises obtaining equations which express the first, second and third distances and determining minima for the equations.

5. The method of claim 1, further comprising:

enhancing a video signal to depict a path comprising the instantaneous position of the moving object.

6. The method of claim 1, wherein:

the third time point (tA1) is after the second time point (tB0) and the second time point is after the first time point (tA0).

7. The method of claim 6, wherein:

the time point at which the instantaneous position of the moving object is determined is between the first and third time points.

8. The method of claim 1, wherein:

the second time point is after the third time point and the third time point is after the first time point.

9. The method of claim 1, wherein:

the third time point is after the first time point and the first time point is after the second time point.

10. A system for determining a position of a moving object used at an event, comprising:

a plurality of cameras capturing first, second and third images of the moving object, the first, second and third images are captured at first, second and third successive time points, the plurality of cameras comprise a first camera which captures the first and third images and a second camera which captures the second image asynchronously to the first camera capturing the first and third images; and

a processor which receives the first, second and third images from the plurality of cameras, the processor;

determines a location of the moving object in the first image in a pixel space of the first camera, and a first vector which extends from an origin of a coordinate system of the first camera to the location of the moving object in the pixel space of the first camera, determines a location of the moving object in the second image in a pixel space of the second camera, and a second vector which extends from an origin of a coordinate system of the second camera to the location of the moving object in the pixel space of the second camera, determines a location of the moving object in the third image in the pixel space of the first camera, and a third vector which extends from an origin of the coordinate system of the first camera to the location of the moving object in the pixel space of the first camera, transforms, to a world coordinate system, the first and third vectors from the coordinate system of the first camera, and the second vector from coordinate system of the second camera, and, to determine an instantaneous position of the moving object in the world coordinate system at a time point which is different than the first, second and third time points, determines a position in 3-d space which satisfies a metric, the metric is based on a first distance from the position in 3-d space to a first line represented by the first vector, a second distance from the position in 3-d space to a second line represented by the second vector, and a third distance from the position in 3-d space to a third line represented by the third vector.

11. The system of claim 10, wherein:

the metric is based on minimizing a sum of the distances.

12. The system of claim 10, wherein:

the metric is based on minimizing a sum of squares of the distances.

13. The system of claim 10, wherein:

the processor enhances a video signal to depict the instantaneous position of the moving object.

14. A processor readable storage device comprising processor readable code embodied thereon for programming a processor to perform a method, the method comprising:

determining a location of a moving object in a first image in a pixel space of a first camera, and a first vector which extends from an origin of a coordinate system of the first camera to the location of the moving object in the pixel space of the first camera;

determining a location of the moving object in a second image in a pixel space of a second camera, and a second vector which extends from an origin of a coordinate system of the second camera to the location of the moving object in the pixel space of the second camera;

determining a location of the moving object in a third image in the pixel space of the first camera, and a third vector which extends from an origin of the coordinate system of the first camera to the location of the moving object in the pixel space of the first camera;

transforming, to a world coordinate system, the first and third vectors from the coordinate system of the first camera, and the second vector from coordinate system of the second camera; and

determining an instantaneous position of the moving object in the world coordinate system, by determining a position in 3-d space which satisfies a metric, the metric is based on a first distance from the position in 3-d space to a first line represented by the first vector, a second distance from the position in 3-d space to a second line represented by the second vector, and a third distance from the position in 3-d space to a third line represented by the third vector.

15. The processor readable storage device of claim 14, wherein:

the first camera captures the first and third images at first and third time points, respectively;

the second camera captures the second image at a second time point, asynchronously to the first camera capturing the first and third images; and

the time point at which the instantaneous position of the moving object is determined is between the first and third time points.

16. The processor readable storage device of claim 14, wherein:

the determining the position in 3-d space comprises obtaining equations which express the first, second and third distances and determining minima for the equations.

17. The processor readable storage device of claim 14, wherein:

the metric is based on minimizing a sum of the distances.

18. The processor readable storage device of claim 14, wherein:

the metric is based on minimizing a sum of squares of the distances.

19. The processor readable storage device of claim 14, wherein:

the first and second images are captured by the first camera before the third image is captured by the second camera.

20. The processor readable storage device of claim 14, wherein:

the second image is captured by the second camera after the first image is captured by the first camera, and before the third image is captured by the first camera.