Method for object tracking

• US 20090080701A1
• Filed: 08/29/2008
• Published: 03/26/2009
• Est. Priority Date: 09/20/2007
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Meuter, Mirko, Iurgel, Uri
• Granted Patent: US 8,055,020 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

The present invention relates to a method for the recognition and tracking of a moving object, in particular of a pedestrian, from a motor vehicle, at which a camera device is arranged. An image of the environment including picture elements is taken in the range of view of the camera device (20) by means of the camera device at regular time intervals and those picture elements are identified with the help of an image processing system which correspond to moving objects to be tracked. A picture element is extracted for each of these objects which represents a projection in image coordinates of that spatial point at which the object contacts a road plane The movement of the corresponding spatial point in the road plane is tracked by means of a state estimator which uses an at least four-dimensional state vector whose components are a position of the spatial point in the road plane and an associated speed in the road plane, wherein the tracking of the movement by the state estimator includes the steps that a prediction is generated for the state vector, this prediction is converted into image coordinates via suitable projection equations, an error to be expected for this prediction is calculated in image coordinates by means of a covariance matrix, and this prediction is compared with the picture element extracted in a later image and is updated.

Claims

1. A method for recognizing and tracking of a moving object from a motor vehicle having a camera device arranged thereon, the method comprising:

taking, using the camera device, images of an environment within a range of view of the camera device at time intervals, said images including picture elements;

identifying for each image, with the aid of an image processing system, the picture elements in the image that correspond to a moving object to be tracked, and extracting a picture element that represents a projection in image coordinates of a spatial point where the object contacts a road plane; and

tracking movement of the spatial point in the road plane using a state estimator that includes an at least four-dimensional state vector comprising a position of the spatial point in the road plane and an associated speed in the road plane;

said tracking comprising the steps of;

generating a prediction for the state vector;

converting the prediction into image coordinates by suitable projection equations;

calculating an error for the prediction in image coordinates by using a covariance matrix; and

comparing the prediction with the picture element extracted in a later image, and

updating the prediction based upon the comparison.

2. A method in accordance with claim 1, wherein the projection equations correspond to a projection of the road plane into an image plane computed using a pinhole camera.

3. A method in accordance with claim 1, wherein the state estimator comprises a Kalman filter.

4. A method in accordance with claim 1, wherein the state estimator uses a linear system equation to compute an estimated value for the state vector.

5. A method in accordance with claim 4, wherein the system equation is composed of a uniform movement of the spatial point in the road plane with the speed and a term that takes into account a translation and a rotation of the vehicle.

6. A method in accordance with claim 5, wherein the translation and the rotation of the vehicle are determined from data generated from a speed sensor and a yaw rate sensor.

7. A method in accordance with claim 5, wherein the translation and the rotation of the vehicle data is transformed into a coordinate system fixed with respect to the vehicle.

8. A method in accordance with claim 1, wherein a direction beam is determined for the extracted picture elements that includes the spatial points in the range of view that were projected onto the image of the respective picture element, and that the spatial point on the directional beam where the object contacts the road plane is determined using camera parameters of the camera device.

9. A method in accordance with claim 4, wherein the covariance matrix of the state vector comprises a term dependent on the system equation, a covariance matrix calculated in a preceding step and a noise matrix, said noise matrix modeling additive white acceleration noise.

10. A method in accordance with claim 9, wherein the covariance matrix corresponding to the error is composed of a constant noise covariance matrix and a variable covariance matrix, said variable covariance matrix computed by transformation from the covariance matrix of the state vector.

11. A method in claim 1, wherein the state estimator comprises a Kalman filter, and wherein during initialization of said Kalman filter, an initial error in the image plane is estimated and transformed to the road plane, and the result of the transformation is used to initialize terms for a position error in the covariance matrix, whereas terms for the variance of the speed in x and y directions is initialized with constant values and is considered statistically independent of the position error.

12. A data processing device comprising a media including computer-readable instructions for recognizing and tracking of a moving object from a motor vehicle having a camera device arranged thereon, said computer-readable instruction being adapted to configure the data processing device to carry out a method comprising:

taking, using the camera device, images of an environment within a range of view of the camera device at time intervals, said images including picture elements;

identifying for each image, with the aid of an image processing system, the picture elements in the image that correspond to a moving object to be tracked, and extracting a picture element that represents a projection in image coordinates of a spatial point where the object contacts a road plane; and

tracking movement of the spatial point in the road plane using a state estimator that includes an at least four-dimensional state vector comprising a position of the spatial point in the road plane and an associated speed in the road plane;

said tracking comprising the steps of;

generating a prediction for the state vector;

converting the prediction into image coordinates by suitable projection equations;

calculating an error for the prediction in image coordinates by using a covariance matrix; and

comparing the prediction with the picture element extracted in a later image, and

updating the prediction based upon the comparison.

13. An apparatus for the recognition and tracking of a moving object from a motor vehicle, said apparatus comprising

a camera device arranged for taking an image, and

a data processing device configured to carry out a method comprising:

taking, using the camera device, images of an environment within a range of view of the camera device at time intervals, said images including picture elements;

identifying for each image, with the aid of an image processing system, the picture elements in the image that correspond to a moving object to be tracked, and extracting a picture element that represents a projection in image coordinates of a spatial point where the object contacts a road plane; and

tracking movement of the spatial point in the road plane using a state estimator that includes an at least four-dimensional state vector comprising a position of the spatial point in the road plane and an associated speed in the road plane;

said tracking comprising the steps of;

generating a prediction for the state vector;

converting the prediction into image coordinates by suitable projection equations;

calculating an error for the prediction in image coordinates by using a covariance matrix; and

comparing the prediction with the picture element extracted in a later image, and

updating the prediction based upon the comparison.