Method for evaluating an object recognition device of a motor vehicle

• US 9,286,525 B2
• Filed: 07/03/2012
• Issued: 03/15/2016
• Est. Priority Date: 07/15/2011
• Status: Active Grant

• Current Assignee: Audi AG (Volkswagen AG)
• Original Assignee: Audi AG (Volkswagen AG)
• Inventors: Nentwig, Mirko
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Williams, Jeffery
• Application Number:
  US13/541,140
• Publication Number:
  US 20130016216A1
• Time in Patent Office: 1,351 Days
• Field of Search: 348/148
• US Class Current: 1/1
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

For testing an object recognition device for a motor vehicle at reasonable costs for different routes, image data for testing the object recognition device may be generated with a camera simulation device. Because the image data of a camera simulation device are artificially generated, it must be made certain that they have a realistic effect on the object recognition device. Reference image data are generated with a camera and simulation image data are generated with the camera simulation device for at least one route. The simulation image data and the reference image data are compared with each other based on at least two comparison measures. A value which is independent of the object recognition device to be tested can be determined for each of the comparison measures. It is then checked if the totality of the generated comparison values satisfies a predetermined validation criterion.

Claims

1. A method for evaluating operability of an object recognition device of a motor vehicle with a camera simulation device, comprising the steps of:

providing reference image data generated with a camera for at least one route;

generating simulation image data for the at least one route with the camera simulation device;

comparing the simulation image data with the reference image data using at least three different predetermined comparison measures

determining a value for each of the at least three comparison measures independent of the processing of the image data by the object recognition device;

forming at least one comparison value for each comparison measure;

checking whether a totality of the comparison values formed for the respective comparison measures satisfies a predetermined validation criterion; and

providing the camera simulation device for use in evaluating the operability of the object recognition device when the predetermined validation criterion is satisfied.

2. The method of claim 1, wherein one of the comparison measures is a direction measure that describes a difference in color intensity data, brightness data, contrast data or edge contour direction data.

3. The method of claim 1, wherein the simulation image data are compared with the reference image data for individual, mutually corresponding image regions of two images, with one of the images being represented by the simulation image data and the other image by the reference image data, and wherein a comparison value is computed for each comparison measure in image region.

4. The method of claim 1, further comprising the steps of:

computing for at least a portion of the simulation image data a first statistical average value and/or a first statistical variance value,

computing for a corresponding portion of the reference image data a second statistical average value and/or a second statistical variance value, and

forming one of the comparison values using the computed statistical values.

5. The method of claim 1, further comprising the steps of:

computing a first histogram for at least a portion of the simulation image intensity gradient direction data,

computing a second histogram for a corresponding portion of the reference image intensity gradient direction data, and

an intensity gradient direction comparison values using the computed histograms.

6. The method of claim 1, wherein the step of checking if the predetermined validation criterion is satisfied, further comprises linking comparison values formed for different comparison measures with each other to form an overall comparison value.

7. The method of claim 6, wherein before linking, at least one of the comparison values is scaled with a predetermined weighting factor for improved for lane departure prevention.

8. The method of claim 1, further comprising iteratively adapting operation of the camera simulation device for use in the evaluation by

defining the operation of the camera simulation device using a configuration data set,

determining whether the predetermined validation criterion is satisfied by the configuration data set, and

changing the configuration data set in response to the validation result.

9. The method of claim 1, further comprising the steps of:

graphically representing at least a portion of the comparison values on a display device, and

locating the displayed comparison values in the image regions that provide the simulation and reference image data used to calculate the respective displayed comparison values.

10. The method of claim 1, wherein the validation criterion is defined using validation image data which are known to result in meaningful evaluation results when evaluating the operability of the object recognition device.

11. The method of claim 1, further comprising the steps of:

defining a comparison value range, and

defining a validation criterion for the defined comparison value range, such that each comparison value from a predetermined set of comparison values or a linkage of the comparison values from the predetermined set is located inside the value range.

19. The method of claim 11 wherein the comparison measures are distance measures and the comparison value range is defined by setting a limit value.

12. Apparatus for testing suitability of a camera simulation device for use in evaluating operability of an object recognition device of a motor vehicle, comprising:

at least three comparison devices, each device configured to compare simulation image data of the camera simulation device with reference image data of a camera using a respective predetermined comparison value so that the comparison value determined independent of processing of the simulation image data and the reference image data by the object recognition device; and

an evaluation device operatively coupled with the at least three comparison devices for checking if a totality of comparison values generated by the comparison devices for the simulation image data satisfies a validation criterion.

13. The apparatus of claim 12 wherein the comparison devices compare the simulation image data with the reference image data for individual, mutually corresponding image regions of the three images, with one of the images being represented by the simulation image data and the other image by the reference image data, and a comparison value is calculated for each image region and for each distance measure.

14. The apparatus of claim 12, wherein evaluation device links comparison values formed for different distance measures with each other to calculate an overall comparison value.

15. The apparatus of claim 14, wherein before linking, at least one of the comparison values is scaled with a predetermined weighting factor.

16. The apparatus of claim 12, further comprising a display device graphically representing at least a portion of the comparison values and associating the displayed comparison values with image regions corresponding to simulation and reference image data used to form the displayed comparison values.

17. The apparatus of claim 12 wherein the evaluation device uses a validation criterion that is defined using validation image data that have resulted in meaningful evaluation results when evaluating the operability of the object recognition device.

18. The apparatus of claim 12 wherein the evaluation device defines a value range, and a validation criterion for the defined value range such that each comparison value from a predetermined set or from a link of the comparison values from the predetermined set is located inside the value range.

20. A method for evaluating the operability of a camera simulation device for testing an object recognition device of a motor vehicle, comprising the steps of:

providing reference image data generated with a camera for at least one route;

generating simulation image data for the at least one route with the camera simulation device;

comparing the simulation image data with the reference image data using at least three different predetermined comparison measures;

determining a value for each of the at least three comparison measures independent of the processing of the data by the object recognition device;

computing at least one comparison value for each comparison measure; and

checking whether a totality of the values computed for the respective comparison measures satisfies a predetermined validation criterion for the camera simulation device.