Object recognition processing apparatus, object recognition processing method, and autonomous driving system

• US 10,460,467 B2
• Filed: 05/23/2017
• Issued: 10/29/2019
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Akiyama, Tomohiro
• Primary Examiner(s): Entezari, Michelle M
• Application Number:
  US15/602,172
• Publication Number:
  US 20170358102A1
• Time in Patent Office: 889 Days
• Field of Search: None
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An object recognition processing apparatus includes: an object fusion processing unit 6 for fusing detection information signals of objects existing around a vehicle, which are detected by a plurality of object detection units to output object fusion signals; a same object fusion result combination candidate selection processing unit for selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined; a same object fusion result combination determination processing unit for determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object; and a same object fusion combination integration processing unit for recognizing the objects existing around the vehicle on the basis of a result of the determination.

Claims

1. An object recognition processing apparatus comprising:

a processor for executing a program; and

a storage device in which the program is stored, wherein the processor executes the program to;

receive a first detection information signal from a first object detection unit and a second detection information signal from a second object detection unit that is different from the first object detection unit;

determine whether or not both the first detection information signal and the second detection information signal have object identification information;

determine whether first object identification information in the first detection information signal is different from second object identification information in the second detection information signal if both the first detection information signal and the second detection information signal have object identification information;

based on the first object identification information being determined to not be different from the second object identification information, set an object motion difference threshold to a first value;

based on one of the first detection information signal and the second detection information signal having object identification information, or none of the first detection information signal and the second detection information signal having object identification information, set the object motion difference threshold to a second value that is less than the first value;

determine whether a difference between first position or speed information of the first detection information signal and second position or speed information of the second detection information signal is equal to or less than the object motion difference threshold; and

based on the difference between the first position or speed information and the second position or speed information being determined to be equal to or less than the object motion difference threshold;

fuse the first and second detection information signals into one fused signal for a same object; and

recognize the same object existing around the vehicle on the basis of the fused signal for the same object.

2. The object recognition processing apparatus according to claim 1, wherein

a combination of object fusion signals, having higher detection reliability of object detection information, is extracted from selected candidates of the object fusion signals, and

it is determined whether or not the combination of the extracted object fusion signals is a combination of object fusion signals of the same object.

3. The object recognition processing apparatus according to claim 1, wherein for a combination of object fusion signals, candidates of the object fusion signals to be combined are selected on the basis of detection reliability condition signals received from the object detection units.

4. The object recognition processing apparatus according to claim 3, wherein object fusion signals satisfying detection reliability conditions included in the detection reliability condition signals received from the object detection units are selected as the candidates.

5. The object recognition processing apparatus according to claim 3, wherein the detection reliability conditions include at least one of:

a number of times by which detection information signals of objects cannot be fused, detection positions of the objects, detection reliabilities of the objects, or estimation errors of detection of the objects.

6. The object recognition processing apparatus according to claim 4, wherein the detection reliability conditions include at least one of:

a number of times by which detection information signals of objects cannot be fused, detection positions of the objects, detection reliabilities of the objects, or estimation errors of detection of the objects.

7. The object recognition processing apparatus according to claim 1, wherein for a combination of object fusion signals, candidates of the object fusion signals to be combined are selected on the basis of object motion specification difference conditions.

8. The object recognition processing apparatus according to claim 7, wherein the object motion specification difference conditions include, regarding a result of fusion of a plurality of objects, at least one of:

a difference in position between the objects, a difference in speed between the objects, or a difference in acceleration between the objects.

9. The object recognition processing apparatus according to claim 7, wherein when an object feature information signal is added to each of the object fusion signals and the object fusion signals are outputted, the object motion specification difference conditions are changed on the basis of object identification information.

10. The object recognition processing apparatus according to claim 7, wherein when an object feature information signal is added to each of the object fusion signals and the object fusion signals are outputted, for a combination of object fusion signals, candidates of the object fusion signals to be combined are selected on the basis of the object feature information signals.

11. The object recognition processing apparatus according to claim 9, wherein the object feature information signal includes at least one of a direction, identification information, a color, or a reception level, of the corresponding object.

13. The object recognition processing apparatus according to claim 1, wherein when determining whether or not a combination of selected object fusion signals is a combination of object fusion signals of the same object, if, with respect to a combination of the object fusion signals selected in latest n processing periods (n:

natural number), a plurality of detection information signals of objects used for fusion of objects for the latest n processing periods are all detected by the same object detection unit, the determination as to whether or not this combination is a combination of object fusion signals of the same object is not performed.

14. The object recognition processing apparatus according to claim 1, wherein when determining whether or not a combination of selected object fusion signals is a combination of object fusion signals of the same object, if the object detection units, which have generated combination candidates of the selected object fusion signals, detect combination candidates of fusion results of the same object at a same timing, the determination as to whether or not the combination candidates are object fusion signals of the same object is not performed.

15. The object recognition processing apparatus according to claim 1, wherein when recognizing an object existing around a vehicle, recognition is performed on a basis of an average value of a plurality of object fusion signals, or estimation error values of the object fusion signals.

16. The object recognition processing apparatus according to claim 1, wherein when recognizing an object existing around a vehicle, an average value of a plurality of object fusion signals or estimation error values of the object fusion signals are determined on the basis of an object identification information signal added to each of the object fusion signals, and the recognition is performed on the basis of the determined value.

17. The object recognition processing apparatus according to claim 15, wherein when recognizing the object existing around the vehicle, the recognition is performed by using the object fusion signals for latest n processing periods.

18. The object recognition processing apparatus according to claim 16, wherein when recognizing objects existing around the vehicle, the recognition is performed by using the object fusion signals for latest n processing periods.

19. An autonomous driving system comprising a vehicle control unit for controlling an operation of a vehicle on the basis of an output of the object recognition processing apparatus according to claim 1.

12. An object recognition processing apparatus for detecting objects existing around a vehicle by using a plurality of object detection units, and receiving detection information of the objects from the respective object detection units, the apparatus comprising:

a processor for executing a program; and

a storage device in which the program is stored, the processor executing the program to perform the operations of;

fusing detection information signals of the objects detected by the plurality of object detection units to output object fusion signals;

selecting, for a combination of object fusion signals, candidates of the object fusion signals to be combined;

determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object; and

recognizing the objects existing around the vehicle on the basis of a result of the determination, wherein when determining whether or not the combination of the selected object fusion signals is a combination of object fusion signals of the same object, if, in a combination of the object fusion signals selected in latest n processing periods (n;

natural number), the number of times of object detections used for fusion of objects for the latest n processing periods is equal to or more than a determination threshold for combination of fusion signals of the same object, this combination is determined to be a combination of fusion results of the same object.

20. An object recognition method, comprising:

receiving a first detection information signal from a first object detection unit and a second detection information signal from a second object detection unit that is different from the first object detection unit;

determining whether or not both the first detection information signal and the second detection information signal have object identification information;

determining whether first object identification information in the first detection information signal is different from second object identification information in the second detection information signal if both the first detection information signal and the second detection information signal have object identification information;

based on the first object identification information being determined to not be different from the second object identification information, setting an object motion difference threshold to a first value;

based on one of the first detection information signal and the second detection information signal having object identification information, or none of the first detection information signal and the second detection information signal having object identification information, setting the object motion difference threshold to a second value that is less than the first value;

determining whether a difference between first position or speed information of the first detection information signal and second position or speed information of the second detection information signal is equal to or less than the object motion difference threshold; and

based on the difference between the first position or speed information and the second position or speed information being determined to be equal to or less than the object motion difference threshold;

fuse the first and second detection information signals into one fused signal for a same object; and

recognize the same object existing around the vehicle on the basis of the fused signal for the same object.

21. An object recognition method comprising:

detecting objects existing around a vehicle by using a plurality of object detectors,

receiving detection information of the objects from the respective object detectors,

fusing detection information signals of the objects detected by the plurality of object detectors to output object fusion signals;

selecting for a combination of object fusion signals, candidates of the object fusion signals to be combined;

determining whether or not a combination of the selected object fusion signals is a combination of object fusion signals of the same object; and

recognizing the objects existing around the vehicle on the basis of a result of the determination, wherein when determining whether or not the combination of the selected object fusion signals is a combination of object fusion signals of the same object, if, in a combination of the object fusion signals selected in latest n processing periods (n;

natural number), the number of times of object detections used for fusion of objects for the latest n processing periods is equal to or more than a determination threshold for combination of fusion signals of the same object, this combination is determined to be a combination of fusion results of the same object.