METHOD AND A DEVICE FOR GENERATING A CONFIDENCE MEASURE FOR AN ESTIMATION DERIVED FROM IMAGES CAPTURED BY A CAMERA MOUNTED ON A VEHICLE

• US 20170278014A1
• Filed: 03/23/2017
• Published: 09/28/2017
• Est. Priority Date: 03/24/2016
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Lessmann, Stephanie, Meuter, Mirko, Westerhoff, Jens
• Granted Patent: US 10,452,999 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 30/1916 Validation; Performance evaluation
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A method of generating a confidence measure for an estimation derived from images captured by a camera mounted on a vehicle includes: capturing consecutive training images by the camera while the vehicle is moving; determining ground-truth data for the training images; computing optical flow vectors from the training images and estimating a first output signal based on the optical flow vectors for each of the training images, the first output signal indicating an orientation of the camera; classifying the first output signal for each of the training images as a correct signal or a false signal depending on how good the first output signal fits to the ground-truth data; determining optical flow field properties for each of the training images derived from the training images; and generating a separation function that separates the optical flow field properties into two classes based on the classification of the first output signal.

Claims

1. A method of generating a confidence measure for an estimation derived from images captured by a camera mounted on a vehicle, the method comprising:

capturing consecutive training images by the camera while the vehicle is moving;

determining ground-truth data for the training images;

computing optical flow vectors from the training images and estimating a first output signal based on the optical flow vectors for each of the training images, the first output signal indicating an orientation of the camera;

classifying the first output signal for each of the training images as a correct signal or a false signal depending on how well the first output signal fits to the ground-truth data;

determining optical flow field properties for each of the training images derived from the training images; and

generating a separation function that separates the optical flow field properties into two classes based on the classification of the first output signal.

2. The method as claimed in claim 1, comprising

capturing consecutive images during runtime by the camera;

computing optical flow vectors from the images captured during runtime and estimating a second output signal based on the optical flow vectors for each of the images, the second output signal indicating an orientation of the camera;

determining optical flow field properties for each of the images derived from the images; and

generating a confidence signal for each of the second output signals by using the determined optical flow field properties and the separation function.

3. The method as claimed in claim 2, comprising

filtering at least some of the second output signals, wherein the confidence signal for each of the second output signals is used as a weighting factor of the respective second output signal.

4. The method as claimed in claim 2, comprising

filtering at least some of the second output signals, wherein the confidence signal for each of the second output signals is used to decide whether the respective second output signal is used for filtering.

5. The method as claimed in claim 3, comprising

determining the orientation of the camera using results of the filtering step.

6. The method as claimed in claim 1, wherein determining the optical flow field properties from the training images includes

selecting a number of points from one of the training images and corresponding points from the subsequent training image;

defining optical flow vectors connecting the points in the one training image and the corresponding points in the subsequent training image;

defining optical flow vectors for other training images;

calculating an error for each of the optical flow vectors; and

writing the optical flow vectors into a histogram for each training image, the histogram representing the distribution of the calculated errors of the optical flow vectors.

7. The method as claimed in claim 2, wherein deriving the optical flow field properties from the images captured during runtime includes

selecting a number of points from one of the images and corresponding points from the subsequent image;

defining optical flow vectors connecting the points in the one image and the corresponding points in the subsequent image;

defining optical flow vectors for other images;

calculating an error for each of the optical flow vectors; and

writing the optical flow vectors into a histogram for each image, the histogram representing the distribution of the calculated errors of the optical flow vectors.

8. The method as claimed in claim 2, comprising

dividing the optical flow field properties derived from the images during runtime into the two classes by using the separation function;

and

generating the confidence signal based on the optical flow field properties divided into the two classes.

9. The method as claimed in claim 1, wherein one of the two classes contains true positives and the other class contains true negatives.

10. The method as claimed in claim 2, wherein the second output signal comprises ego-motion estimates and the confidence signal indicates ego-motion confidence.

11. The method as claimed in claim 2, wherein the second output signal comprises estimates of a pitch angle and a roll angle of the camera relative to a surface and the confidence signal indicates confidence of the pitch angle and the roll angle.

12. The method as claimed in claim 11, wherein properties to derive the separation function comprise ego-motion confidence, count of homography inliers, ratio of homography inliers to ego-motion inliers, position of centroid of the road plane inliers distribution and/or real world curve radius derived from yaw rate and vehicle velocity.

13. The method as claimed in claim 1, wherein the separation function is generated by means of machine learning techniques.

14. A device for generating a confidence measure for an estimation derived from images captured by a camera mounted on a vehicle, the device is configured to:

receive consecutive training images captured by the camera while the vehicle is moving;

receive ground-truth data for the training images;

compute optical flow vectors from the training images and estimate a first output signal based on the optical flow vectors for each of the training images, the first output signal indicating an orientation of the camera;

classify the first output signal for each of the training images as a correct signal or a false signal depending on how good the first output signal fits to the ground-truth data;

determine optical flow field properties for each of the training images derived from the training images; and

generate a separation function that separates the optical flow field properties into two classes based on the classification of the first output signal.