Object detection method and apparatus

• US 10,438,489 B2
• Filed: 12/27/2017
• Issued: 10/08/2019
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Baba, Takahiro
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US15/855,803
• Publication Number:
  US 20180182243A1
• Time in Patent Office: 650 Days
• Field of Search: 382103
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/9324 Alternative operation using ultrasonic waves
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

In an object detection apparatus, a first association unit associates, based on radar information and camera information, radar objects with at least one camera object that are estimated to be the same as each other. If camera objects are detected so that the camera information includes at least the detection point and the type of each of the camera objects, a determination unit determines whether the camera objects express a common object in accordance with the camera information. A second association unit defines, upon determination that the camera objects express the common object, the camera objects as a camera object group, and associates, if one of the camera objects constituting the camera object group is associated with a target radar object that is one of the radar objects, all the camera objects constituting the camera object group with the target radar object.

Claims

1. An object detection apparatus comprising:

a first acquisition section configured to acquire radar information that includes at least respective detection points of radar objects, the radar objects being objects detected by a radar sensor based on a detection range;

a second acquisition unit configured to;

perform a comparing task to compare, based on an image captured by a camera that has a capture range which overlaps at least a part of the detection range of the radar sensor, predetermined feature quantities extracted from the image with feature quantity templates;

detect, based on a result of the comparison task, at least one object as a camera object, the feature quantity templates having been prepared beforehand for each of respective types of object to be detected; and

acquire camera information that includes at least a detection point and type of the at least one camera object;

a first association unit configured to associate, based on the radar information and the camera information, at least one of the radar objects with the at least one camera object upon determination that the at least one of the radar objects is estimated to be identical to the at least one camera object;

a determination unit configured to, if a plurality of camera objects are detected so that the camera information includes at least the detection point and the type of each of the camera objects, determine whether the camera objects express a common object in accordance with the camera information; and

a second association unit configured to;

define, upon determination that the camera objects express the common object, the camera objects as a camera object group; and

associate, if one of the camera objects constituting the camera object group is associated with a target radar object that is one of the radar objects, all the camera objects constituting the camera object group with the target radar object.

2. The object detection apparatus according to claim 1, further comprising:

a region setting section configured to;

set a first region for each of the radar objects respectively, the first region including a detection point of the radar object; and

set a second region for each of the camera objects respectively, the second region including a detection point of the camera object,

wherein the determination section is configured to;

determine whether there is mutual overlap between the second regions that have been set to the respective camera objects; and

determine that that the camera objects express the common object if there is mutual overlap between the that have been set to the respective camera objects.

3. The object detection apparatus according to claim 1, wherein:

the camera information about each of the camera objects includes, as a determination-use information item, at least one of;

a distance to the camera object relative to the object detection apparatus;

a direction to the position of the camera object relative to the object detection apparatus, and

a width of the camera object; and

the determination unit is configured to determine that the camera objects express the common object upon determination that a difference value is less than a predetermined difference value threshold value, the difference value expressing an amount of difference between the determination-use information items of the respective camera objects.

4. The object detection apparatus according to claim 1, further comprising:

an identifying section configured to, if a single object that is expressed by the target radar object and the camera object group is defined as an associated object, identify a type of the associated object in accordance with the radar information about the target radar object.

5. The object detection apparatus according to claim 4, wherein:

the radar information about each radar object includes at least one of a speed and a speed vector of the radar object; and

the identifying unit is configured to identify the type of the associated object in accordance with at least one of the speed and the speed vector of the target radar object in the associated object.

6. The object detection apparatus according to claim 4, further comprising:

a fusion unit configured to;

execute a sensor fusion task that fuses the detection point of the target radar object included in the associated object and the detection points of a selected one of the camera objects included in the associated object, a type of the selected one of the camera objects corresponding to the type of the associated object identified by the identifying unit.

7. An object detection apparatus comprising:

a first acquisition section configured to acquire radar information that includes at least respective detection points of radar objects, the radar objects being objects detected by a radar sensor based on a detection range;

a second acquisition unit configured to;

perform a comparing task to compare, based on an image captured by a camera that has a capture range which overlaps at least a part of the detection range of the radar sensor, predetermined feature quantities extracted from the image with feature quantity templates;

detect, based on a result of the comparison task, objects as camera objects, the feature quantity templates having been prepared beforehand for each of respective types of object to be detected; and

acquire camera information that includes at least a detection point and type of each of the camera objects;

a determination unit configured to determine whether the camera objects express a common object in accordance with the camera information; and

an association unit configured to;

define, upon determination that the camera objects express the common object, the camera objects as a camera object group;

estimate, based on the radar information and the camera information, whether at least one of the radar objects and at least one of the camera objects are identical to each other; and

associate, upon determination that the camera objects express the camera object group and that a candidate camera object that is one of the camera objects and a selected one of the radar objects have been estimated to be identical to each other, all the camera objects constituting the camera object group with the selected one of the radar objects.

8. The object detection apparatus according to claim 7, wherein:

the association unit is configured to associate the candidate camera object with the selected one of the radar objects upon determination that the camera objects do not express the camera object group and that the candidate camera object and the selected one of the radar objects have been estimated to be identical to each other, the candidate camera object with the selected one of the radar objects.

9. An object detection method comprising:

acquiring radar information that includes at least respective detection points of radar objects, the radar objects being objects detected by a radar sensor based on a detection range;

performing a comparing task to compare, based on an image captured by a camera that has a capture range which overlaps at least a part of the detection range of the radar sensor, predetermined feature quantities extracted from the image with feature quantity templates;

detecting, based on a result of the comparison task, at least one object as a camera object, the feature quantity templates having been prepared beforehand for each of respective types of object to be detected;

acquiring camera information that includes at least a detection point and type of the at least one camera object;

associating, based on the radar information and the camera information, the radar objects with the at least one camera object that are estimated to be the same as each other;

determining, if a plurality of camera objects are detected so that the camera information including at least the detection point and the type of each of the camera objects, whether the camera objects express a common object in accordance with the camera information;

defining, upon determination that the camera objects express the common object, the camera objects as a camera object group; and

associating, if one of the camera objects constituting the camera object group is associated with a target radar object that is one of the radar objects, all the camera objects constituting the camera object group with the target radar object.