Object detection apparatus

• US 10,217,024 B2
• Filed: 09/15/2015
• Issued: 02/26/2019
• Est. Priority Date: 09/24/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Baba, Takahiro
• Primary Examiner(s): Coleman, Stephen P
• Application Number:
  US15/513,520
• Publication Number:
  US 20170300780A1
• Time in Patent Office: 1,260 Days
• Field of Search: None
• CPC Class Codes: B60R 21/00 Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks safety belts or body harnesses in vehicles B60R22/00; devices, apparatus or methods for life-saving in general A62B; safety devices for propulsion unit control specially adapted for, or arranged in, vehicles B60K28/00; seats constructed to protect the occupant from the effect of abnormal g-forces, e.g. crash or safety seats, B60N2/42; energy-absorbing arrangements for hand wheels for steering vehicles B62D1/11; energy-absorbing arrangements for vehicle steering columns B62D1/19; harnessing in aircraft B64D25/00
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 7/41 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

In an object detection apparatus, a first trajectory estimation unit estimates a trajectory of a first object detected by an electromagnetic wave sensor. An optical flow acquisition unit image-processes a captured image acquired from a camera to acquire movement directions based on optical flows of feature points in the captured image. A movement direction match determination unit determines whether or not a match occurs between a movement direction based on the optical flows and a movement direction based on the trajectory of the first object. If a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object, a sameness determination unit determines that a second object identified by the plurality of feature points and the first object are a same object.

Claims

1. An object detection apparatus mounted in a vehicle, comprising:

a first trajectory estimation unit configured to estimate, using a processor, a trajectory of a first object that is an object detected by an electromagnetic wave sensor configured to transmit and receive electromagnetic waves to detect a position of the object;

an optical flow acquisition unit configured to image process, using the processor, a captured image acquired from a camera to acquire movement directions based on optical flows of a large number of feature points in the captured image;

a movement direction match determination unit configured to determine, using the processor, whether or not a match occurs between a movement direction based on optical flows of a plurality of feature points, of the large number of feature points, situated around the periphery of the first object and a movement direction based on the trajectory of the first object; and

a sameness determination unit configured to, if a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object, determine, using the processor, that a second object identified by the plurality of feature points and the first object are a same object,

wherein a collision mitigation control of the vehicle is performed based on the position of the object.

2. The apparatus according to claim 1, further comprising:

a distance acquisition unit configured to acquire, using the processor, a distance to the second object; and

a region determination unit configured to determine, using the processor, whether or not the distance to the second object is less than a predetermined distance threshold and determine whether or not a position of the second object in the captured image is within either one of rightmost and leftmost regions respectively including right and left edges of the captured image,

wherein the movement direction match determination unit is configured to, if the distance to the second object is less than the predetermined distance threshold and if the position of the second object in the captured image is within either one of the rightmost and leftmost regions of the captured image, then determine whether or not a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object.

3. The apparatus according to claim 1, further comprising:

a first region definition unit configured to define, using the processor, a first object region including a first detection point representing a relative position of the first object with respect to a reference point in an XY-plane with an X-axis direction being a vehicle widthwise direction and a Y-axis direction being a vehicle lengthwise direction; and

a second region definition unit configured to define, using the processor, a second object region including a second detection point representing a relative position of the second object with respect to the reference point in the XY-plane,

wherein the sameness determination unit is configured to, if there is an overlap of the first and second object regions in the XY-plane and if a match occurs between the movement direction based on the trajectory of the first object and the movement direction based on the optical flows of the plurality of feature points, determine that the first object and the second object are the same object.

4. The apparatus according to claim 2, further comprising:

a first region definition unit configured to define, using the processor, a first object region including a first detection point representing a relative position of the first object with respect to a reference point in an XY-plane with an X-axis direction being a vehicle widthwise direction and a Y-axis direction being a vehicle lengthwise direction; and

a second region definition unit configured to define, using the processor, a second object region including a second detection point representing a relative position of the second object with respect to the reference point in the XY-plane,

wherein the sameness determination unit is configured to, if the distance to the second object is equal to or greater than the predetermined distance threshold or if the position of the second object is outside the rightmost and leftmost regions of the captured image, and if there is an overlap of the first and second object regions in the XY-plane, then determine that the first object and the second object are the same object.

5. The apparatus according to claim 4, wherein

the sameness determination unit is configured to, if the distance to the second object is less than the predetermined distance threshold, if the position of the second object in the captured image is outside the rightmost and leftmost regions of the captured image, if there is an overlap of the first and second object regions in the XY-plane, and if a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object, determine, using the processor, that a second object identified by the plurality of feature points and the first object are the same object.

6. A method for performing object detection by a vehicle equipped with an electromagnetic wave sensor configured to transmit and receive electromagnetic waves to detect a position of an object and a camera configured to capture images, comprising:

estimating a trajectory of a first object that is an object detected by the electromagnetic wave sensor;

image processing a captured image acquired from the camera to acquire movement directions based on optical flows of a large number of feature points in the captured image;

determining whether or not a match occurs between a movement direction based on optical flows of a plurality of feature points, of the large number of feature points, situated around the periphery of the first object and a movement direction based on the trajectory of the first object;

determining that a second object identified by the plurality of feature points and the first object are a same object when a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object; and

performing a collision mitigation control of the vehicle based on the position of the object.

7. The method according to claim 6, further comprising:

acquiring a distance to the second object; and

determining whether or not the distance to the second object is less than a predetermined distance threshold and determining whether or not a position of the second object in the captured image is within either one of rightmost and leftmost regions respectively including right and left edges of the captured image,

wherein if the distance to the second object is less than the predetermined distance threshold and if the position of the second object in the captured image is within either one of the rightmost and leftmost regions of the captured image, determining whether or not a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object.

8. The method according to claim 6, further comprising:

defining a first object region including a first detection point representing a relative position of the first object with respect to a reference point in an XY-plane with an X-axis direction being a vehicle widthwise direction and a Y-axis direction being a vehicle lengthwise direction; and

defining a second object region including a second detection point representing a relative position of the second object with respect to the reference point in the XY-plane,

wherein if there is an overlap of the first and second object regions in the XY-plane and if a match occurs between the movement direction based on the trajectory of the first object and the movement direction based on the optical flows of the plurality of feature points, determining that the first object and the second object are the same object.

9. The method according to claim 7, further comprising:

defining a first object region including a first detection point representing a relative position of the first object with respect to a reference point in an XY-plane with an X-axis direction being a vehicle widthwise direction and a Y-axis direction being a vehicle lengthwise direction; and

defining a second object region including a second detection point representing a relative position of the second object with respect to the reference point in the XY-plane,

wherein if the distance to the second object is equal to or greater than the predetermined distance threshold or if the position of the second object is outside the rightmost and leftmost regions of the captured image, and if there is an overlap of the first and second object regions in the XY-plane, determining that the first object and the second object are the same object.

10. The method according to claim 9, wherein

if the distance to the second object is less than the predetermined distance threshold, if the position of the second object in the captured image is outside the rightmost and leftmost regions of the captured image, if there is an overlap of the first and second object regions in the XY-plane, and if a match occurs between the movement direction based on the optical flows of the plurality of feature points and the movement direction based on the trajectory of the first object, determining that a second object identified by the plurality of feature points and the first object are the same object.