METHOD AND APPARATUS FOR GENERATING THREE-DIMENSIONAL (3D) ROAD MODEL

• US 20180131924A1
• Filed: 03/07/2017
• Published: 05/10/2018
• Est. Priority Date: 11/07/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: JUNG, Hwiryong, SUNG, Young Hun, LEE, KeeChang, JUNG, Kyungboo
• Granted Patent: US 10,863,166 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3819 Road shape data, e.g. outline of a route
• CPC Class Codes: G01C 21/3822 Road feature data, e.g. slope data
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G01C 21/3867 Geometry of map features, e.g. shape points, polygons or for simplified maps
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0077 Colour aspects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method for generating a three-dimensional (3D) lane model, the method including calculating a free space indicating a driving-allowed area based on a driving image captured from a vehicle camera, generating a dominant plane indicating plane information of a road based on either or both of depth information of the free space and a depth map corresponding to a front of the vehicle, and generating a 3D short-distance road model based on the dominant plane.

Claims

1. A method of generating a three-dimensional (3D) lane model, the method comprising:

calculating a free space indicating a driving-allowed area based on a driving image captured from a vehicle camera;

generating a dominant plane indicating plane information of a road based on either or both of depth information of the free space and a depth map corresponding to a front of the vehicle; and

generating a 3D short-distance road model based on the dominant plane.

2. The method of claim 1, wherein the calculating of the free space comprises:

acquiring the driving image from the camera included in the vehicle;

generating the depth map including depth information corresponding to a visual field criterion of the front of the vehicle; and

calculating the free space based on the driving image and the depth map.

3. The method of claim 2, wherein the acquiring of the driving image comprises acquiring the driving image including any one or any combination of two or more of camera information, a black-and-white image, and a color image captured by the camera.

4. The method of claim 2, wherein the generating of the depth map comprises acquiring the depth information using any one or any combination of two or more of a radar, a lidar, and a stereo matching method using a stereo camera.

5. The method of claim 1, further comprising:

extracting road shape information including a surface of the road, a lane marking on the road, and a center line of the road from the driving image.

6. The method of claim 5, wherein the generating of the 3D short-distance road model comprises:

generating 3D coordinates based on two-dimensional (2D) coordinates of vertices configuring the lane marking and depth information corresponding to the 2D coordinates of the vertices on the dominant plane;

generating 3D lane information by performing inverse projection on the generated 3D coordinates; and

generating the 3D short-distance road model based on the 3D lane information.

7. The method of claim 5, wherein the extracting of the road shape information comprises:

acquiring location information including either one or both of current location information of the vehicle and direction information of the vehicle; and

extracting the road shape information currently viewed by the camera based on the location information and camera information included in the driving image.

8. The method of claim 7, wherein the acquiring of the location information comprises acquiring the location information using any one or any combination of two or more of a global positioning system (GPS), an inertial measurement unit (IMU), and a visual odometry method.

9. The method of claim 7, wherein the acquiring of the location information comprises:

acquiring travel route information based on a starting point and a destination of the vehicle from map information; and

acquiring location information including either one or both of current location information of the vehicle and direction information of the vehicle based on the travel route information.

10. The method of claim 9, wherein the extracting of the road shape information comprises extracting road shape information entering a viewing frustum of the camera from the map information based on the location information and the camera information included in the driving image.

11. The method of claim 10, further comprising:

generating a 3D long-distance road model based on the road shape information and the 3D short-distance road model.

12. The method of claim 11, wherein the generating of the 3D long-distance road model comprises:

matching the road shape information and the 3D short-distance road model; and

projecting a matching result onto the dominant plane and generating the 3D long-distance road model.

13. The method of claim 12, wherein the matching comprises matching the road shape information and the 3D short-distance road model using a least square method (LSM).

14. The method of claim 11, wherein the generating of the 3D long-distance road model comprises:

acquiring height information from the dominant plane; and

generating the 3D long-distance road model based on the height information, the road shape information, and the 3D short-distance road model.

15. A non-transitory computer-readable storage medium storing program instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

16. An apparatus for generating a three-dimensional (3D) road model, the apparatus comprising:

a camera configured to capture a driving image from a vehicle; and

a processor configured to calculate a free space indicating a driving-allowed area based on the driving image, generate a dominant plane indicating plane information of a road based on either one or both of depth information of the free space and a depth map corresponding to a front of the vehicle, and generate a 3D short-distance road model based on the dominant plane.

17. The apparatus of claim 16, wherein the processor is further configured to acquire the driving image from a camera included in the vehicle, generate the depth map including depth information corresponding to a visual field criterion of the front of the vehicle, and calculate the free space based on the driving image and the depth map.

18. The apparatus of claim 16, wherein the processor is further configured to extract road shape information including a surface of the road, a lane marking on the road, and a center line of the road from the driving image, generate 3D coordinates based on two-dimensional (2D) coordinates of vertices configuring the lane marking and depth information corresponding to the 2D coordinates of the vertices on the dominant plane, and generate the 3D short-distance road model based on 3D lane information generated by performing inverse projection on the generated 3D coordinates.

19. The apparatus of claim 18, further comprising:

a memory configured to store map information, and

wherein the processor is configured to acquire location information including either one or both of current location information of the vehicle or direction information of the vehicle based on travel route information generated using a starting point and a destination of the vehicle from the map information stored in the memory, and extract road shape information entering a viewing frustum of the camera from the map information based on the location information and the camera information included in the driving image.

20. The apparatus of claim 19, wherein the processor is configured to match the road shape information and the 3D short-distance road model and project a matching result onto the dominant plane and generating the 3D long-distance road model.