Detecting method and device of obstacles based on disparity map and automobile driving assistance system

• US 10,217,007 B2
• Filed: 03/27/2017
• Issued: 02/26/2019
• Est. Priority Date: 01/28/2016
• Status: Active Grant

• Current Assignee: Beijing Smarter Eye Technology Co., Ltd.
• Original Assignee: Beijing Smarter Eye Technology Co., Ltd.
• Inventors: Xie, Qiwei, Jiang, An, Chen, Xi, Cui, Feng, Zhu, Haitao, Meng, Ran
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/470,283
• Publication Number:
  US 20180018528A1
• Time in Patent Office: 701 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G08G 1/0133 for classifying traffic situation
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

The present disclosure provides a disparity map-based obstacle detection method, a disparity map-based obstacle detection device, and a vehicle assistant driving system. The method includes steps of: acquiring a disparity map and a V disparity map in accordance with an image including a road surface; simulating the road surface in accordance with the V disparity map; identifying a first obstacle in accordance with a simulation result; extracting an object whose disparity value is greater than a first threshold in accordance with the disparity map, and subjecting the object to morphological operation so as to identify a second obstacle; extracting an object whose disparity value is smaller than a second threshold in accordance with the disparity map, and subjecting the object to morphological operation so as to identify a third obstacle; and screening the first obstacle, the second obstacle and the third obstacle in accordance with a training model, so as to detect the obstacle, the training model being acquired through machine learning in accordance with correct obstacle information and erroneous obstacle information. According to the present disclosure, it is able to improve the robustness.

Claims

1. A disparity map-based obstacle detection method, at least comprising the steps of:

acquiring a disparity map and a V-disparity map in accordance with an image comprising a road surface;

simulating the road surface in accordance with the V-disparity map;

identifying a first obstacle in accordance with a result of simulating the road surface;

extracting an object whose disparity value is greater than a first threshold in accordance with the disparity map, and subjecting the extracted object whose disparity value is greater than the first threshold to a morphological operation so as to identify a second obstacle;

extracting an object whose disparity value is smaller than a second threshold in accordance with the disparity map, and subjecting the extracted object whose disparity value is smaller than the second threshold to a morphological operation so as to identify a third obstacle; and

screening the first obstacle, the second obstacle and the third obstacle in accordance with a training model, so as to detect a final obstacle, the training model being acquired through machine learning in accordance with correct obstacle information and erroneous obstacle information.

2. The disparity map-based obstacle detection method according to claim 1, wherein the training model is acquired by the steps of:

acquiring the disparity map and the V-disparity map in accordance with the image comprising the road surface;

simulating the road surface in accordance with the V-disparity map;

identifying the first obstacle in accordance with the result of simulating the road surface;

extracting the object whose disparity value is greater than the first threshold in accordance with the disparity map, and subjecting the extracted object whose disparity value is greater than the first threshold to a the morphological operation so as to identify the second obstacle;

extracting the object whose disparity value is smaller than the second threshold in accordance with the disparity map, and subjecting the extracted object whose disparity value is smaller than the second threshold to a the morphological operation so as to identify the third obstacle;

manually screening the first obstacle, the second obstacle and the third obstacle;

marking the correct obstacle information and the erroneous obstacle information in accordance with a screening result, so as to form a training set; and

performing machine learning in accordance with the training set as well as obstacle region information and an image characteristic in the disparity map, so as to generate the training model.

3. The disparity map-based obstacle detection method according to claim 2, wherein prior to manually screening the first obstacle, the second obstacle, and the third obstacle, gradient information of the disparity map is calculated, and an obstacle having a small gradient modulus value is removed in accordance with the gradient information and disparity information.

4. A disparity map-based obstacle detection device, at least comprising:

an acquisition unit configured to acquire a disparity map and a V-disparity map in accordance with an image including a road surface;

a simulation unit configured to simulate the road surface in accordance with the V-disparity map;

a first identification unit configured to identify a first obstacle in accordance with a result of simulating the road surface;

a second identification unit configured to extract an object whose disparity value is greater than a first threshold in accordance with the disparity map, and subject the extracted object whose disparity value is greater than the first threshold to a morphological operation so as to identify a second obstacle;

a third identification unit configured to extract an object whose disparity value is smaller than a second threshold in accordance with the disparity map, and subject with the extracted object whose disparity value is smaller than the second threshold to a morphological operation so as to identify a third obstacle; and

a detection unit configured to screen the first obstacle, the second obstacle and the third obstacle in accordance with a training model, so as to detect a final obstacle, the training model being acquired through machine learning in accordance with correct obstacle information and erroneous obstacle information.

5. The disparity map-based obstacle detection device according to claim 4, wherein the detection unit comprises:

an acquisition module configured to acquire the disparity map and the V-disparity map in accordance with the image including the road surface;

a simulation module configured to simulate the road surface in accordance with the V-disparity map;

a first identification module configured to identify the first obstacle in accordance with the result of simulating the road surface;

a second identification module configured to extract the object whose disparity value is greater than the first threshold in accordance with the disparity map, and subject the extracted object whose disparity value is greater than the first threshold to a the morphological operation so as to identify the second obstacle;

a third identification module configured to extract the object whose disparity value is smaller than the second threshold in accordance with the disparity map, and subject the extracted object whose disparity value is smaller than the second threshold to a the morphological operation so as to identify the third obstacle;

a screening module configured to manually screen the first obstacle, the second obstacle and the third obstacle;

a marking module configured to mark the correct obstacle information and the erroneous obstacle information in accordance with a screening result, so as to form a training set; and

a generation module configured to perform machine learning in accordance with the training set as well as obstacle region information and an image characteristic in the disparity map, so as to generate the training model.

6. The disparity map-based obstacle detection device according to claim 5, wherein the detection unit further comprises:

a calculation module configured to calculate gradient information of the disparity map; and

a removal module configured to remove an obstacle having a small gradient modulus value in accordance with the gradient information and disparity information.

7. A vehicle assistant driving system, comprising the disparity map-based obstacle detection device according to any one of claims 4 to 6.