Driving assist apparatus, driving assist method, and non-transitory computer readable recording medium storing program

• US 10,192,122 B2
• Filed: 08/21/2014
• Issued: 01/29/2019
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Yoshida, Michinori, Tsushima, Naoyuki
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US15/323,922
• Publication Number:
  US 20170140230A1
• Time in Patent Office: 1,622 Days
• Field of Search: 382103, 382104, 382106, 382153, 382165, 382170, 382171, 382190, 382195, 382199, 382201
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A driving assist apparatus includes an image acquisition part to acquire a captured image around a vehicle, a position information acquisition part to acquire position information of a first object existing around the vehicle and detected by a sensor, a detection range determination part to determine a detection range of the first object within the captured image based on the position information of the first object, and an object recognition part to perform image processing on, within the captured image, an image existing in a range other than the detection range of the first object, thereby to recognize a second object that is different from the first object. Hence, a processing time taken for recognizing the second object from the captured image can be shortened.

Claims

1. A driving assist apparatus comprising:

a memory; and

a processor configured to

acquire, using a camera, a captured image around a vehicle;

acquire, from one or more sensors other than the camera, position information of one or more objects detected by the one or more sensors to exist around the vehicle;

determine a detection range of the one or more sensor detected objects within the captured image based on the acquired position information;

determine one or more regions of the captured image which correspond to the determined detection range of the one or more sensor detected objects; and

perform image processing on a region of the captured image other than a region corresponding to a detection range of an object detected by the one or more sensors to recognize a second object that is different from any sensor detected objects, wherein

the processor is further configured to

extract a pixel group having a color density equal to or higher than a threshold, from a region of the captured image other than the one or more regions corresponding to the detection range of the one or more sensor detected objects;

calculate a detecting region based on the extracted pixel group; and

performing image processing on the calculated detecting region of the captured image to recognize the second object, and

the processor extracts the pixel group having the color density equal to or higher than the threshold from a region of the capture image in a traffic lane included in the captured image.

2. The driving assist apparatus according to claim 1, wherein the memory stores an edge image expressing a feature of an object to be detected, and

recognizing whether a second object is in the calculated detecting region of the captured image by matching the image within the calculated detecting region with the stored edge image.

3. The driving assist apparatus according to claim 1, wherein the processor extracts the pixel group having the color density equal to or higher than the threshold from a region within the captured image existing below a horizontal-direction line at such a height where a vanishing point is included.

4. The driving assist apparatus according to claim 1, wherein the processor is further configured to generate a model that retains distance information of a distance between the vehicle and the recognized second object.

5. The driving assist apparatus according to claim 4, wherein when distances among a plurality of generated models are each equal to or lower than a threshold, the processor stores the plurality of models as a single model.

6. A driving assist apparatus comprising:

a memory; and

a processor configured to

acquire, using a camera, a captured image around a vehicle;

acquire, from one or more sensors other than the camera, position information of at least two objects detected by the one or more sensors to exist around the vehicle, each of the at least two objects being detected by radiation with one of the one or more sensors in a first-line direction;

determine a first detection range of a first object of the at least two radiation detected objects within the captured image based on the acquired position information of the at least two radiation detected objects,

determine a second detection range of a second object of the at least two radiation detected objects within the captured image based on the acquired position information of the at least two radiation detected objects;

determine a region within the captured image which corresponds to a composite of the determined first and second detection ranges; and

perform image processing on a region of the captured image other than a region determined to correspond to the composite range of the first and second detection ranges of the at least two radiation detected objects to recognize a third object that is different from a objects detected by the one or more sensors, wherein

the processor is further configured to

extract a pixel group having a color density equal to or higher than a threshold, from a region of the captured image other than the one or more regions corresponding to the detection range of the one or more sensor detected objects;

calculate a detecting region based on the extracted pixel group; and

performing image processing on the calculated detecting region of the captured image to recognize the second object, and

the processor extracts the pixel group having the color density equal to or higher than the threshold from a region of the capture image in a traffic lane included in the captured image.

7. A driving assist method comprising:

acquiring, using a camera, a captured image around a vehicle;

acquiring;

from one or more sensors, position information of one or more objects detected by the one or more sensors to exist around the vehicle;

determining, using a processor, a detection range of the one or more sensor detected objects within the captured image based on the acquired position information;

determining one or more regions of the captured image which correspond to the determined detection range of the one or more sensor detected objects;

extracting a pixel group having a color density equal to or higher than a threshold, from a region of the captured image in a traffic lane included in the captured image and other than the one or more regions corresponding to the detection range of the one or more sensor detected objects;

calculating a detecting region based on the extracted pixel group;

performing image processing on the calculated detecting region to recognize a second object that is different from any sensor detected objects.

8. A non-transitory computer readable recording medium storing a program that when executed causes a computer to:

acquire, using a camera, a captured image around a vehicle;

acquire, from one or more sensors other than the camera, position information of one or more objects detected by the one or more sensors to exist around the vehicle;

determine a detection range of the one or more sensor detected objects within the captured image based on the acquired position information;

determine one or more regions of the captured image which correspond to the determined detection range of the one or more sensor detected objects;

extract a pixel group having a color density equal to or higher than a threshold, from a region of the captured image in a traffic lane included in the captured image and other than the one or more regions corresponding to the detection range of the one or more sensor detected objects;

calculate a detecting region based on the extracted pixel group; and

perform image processing on the calculated detecting region to recognize a second object that is different from any sensor detected objects.