Crowd-sensed point cloud map

• US 10,529,089 B2
• Filed: 02/23/2018
• Issued: 01/07/2020
• Est. Priority Date: 02/23/2018
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company), University of Southern California
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Ahmad, Fawad, Qiu, Hang, Bai, Fan, Govindan, Ramesh
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US15/903,616
• Publication Number:
  US 20190266748A1
• Time in Patent Office: 683 Days
• Field of Search: None
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

Systems and method are provided for controlling an autonomous vehicle. In one embodiment, a method includes: receiving sensor data from a sensor of the vehicle; determining a three dimensional point cloud map segment from the sensor data; determining a vehicle pose associated with the three-dimensional point cloud map segment; determining a pose difference based on the vehicle pose, another vehicle pose, and a two-step process, wherein the two-step process includes computing a coarse-granularity pose difference, and computing a fine-granularity pose difference; aligning the three dimensional point cloud map segment with another three dimensional point cloud map segment associated with the other vehicle pose based on the pose difference; and controlling the vehicle based on the aligned three dimensional point cloud map segments.

Claims

1. A method for controlling an autonomous vehicle, comprising:

receiving sensor data from a sensor of the vehicle;

determining a three dimensional point cloud map segment from the sensor data;

determining a vehicle pose associated with the three-dimensional point cloud map segment;

determining a pose difference based on the vehicle pose, another vehicle pose, and a two-step process, wherein the two-step process includes computing a coarse-granularity pose difference, and computing a fine-granularity pose difference;

aligning the three dimensional point cloud map segment with another three dimensional point cloud map segment associated with the other vehicle pose based on the pose difference; and

controlling the vehicle based on the aligned three dimensional point cloud map segments.

2. The method of claim 1, wherein the coarse-granularity pose difference is computed based on a mean square error between two trajectories.

3. The method of claim 2, wherein the fine-granularity pose difference is computed based on an image-plane re-projection error minimization method.

4. The method of claim 3, further comprising determining transformation data based on an iterative closest point convergence method using the fine-granularity pose difference, and wherein the aligning is based on the transformation data.

5. The method of claim 1, wherein the determining the vehicle pose is based on a two dimensional feature set from the sensor data, a three dimensional point cloud associated with the two dimensional feature set, and an image-plane re-projection error minimization method.

6. The method of claim 5, wherein the two dimensional feature set is associated with static features, and wherein the three dimensional point cloud is associated with static features.

7. The method of claim 1, wherein the determining the vehicle pose is based on a first two dimensional feature set from the sensor data, a second two dimensional feature set from the sensor data, a first three dimensional point cloud associated with the first two dimensional feature set, a second three dimensional point cloud associated with the second two dimensional feature set, and an image-plane re-projection error minimization method.

8. The method of claim 7, where the first two dimensional feature set is associated with static features, wherein the first three dimensional point cloud is associated with static features, wherein the second two dimensional feature set is associated with dynamic features, and wherein the second three dimensional point cloud is associated with dynamic features.

9. The method of claim 1, further comprising merging features of the aligned three dimensional point cloud map segments based on a confidence of the feature.

10. The method of claim 9, wherein the merging the features is further based on a multi-partite matching method.

11. The method of claim 1, further comprising localizing the vehicle based on the aligned three dimensional point cloud map segments and a mutli-partite matching method.

12. A computer-implemented system for controlling an autonomous vehicle, comprising:

a non-transitory computer readable medium comprising;

a map segment generation module configured to receive sensor data from a sensor of the vehicle, and determine a three dimensional point cloud map segment from the sensor data;

a pose determination module configured to determine a vehicle pose associated with the three-dimensional map segment;

a pose difference determination module configured to determine a pose difference based on the vehicle pose, another vehicle pose, and a two-step process, wherein the two-step process includes computing a coarse-granularity pose difference, and computing a fine-granularity pose difference;

an alignment module configured to align the three dimensional point cloud map segment with another three dimensional map segment associated with the other vehicle pose based on the pose difference; and

a control module configured to control the vehicle based on the aligned three dimensional point cloud map segments.

13. The system of claim 12, wherein the coarse-granularity pose difference is computed based on a mean square error between two trajectories.

14. The system of claim 13, wherein the fine-granularity pose difference is computed based on an image-plane re-projection error minimization method.

15. The system of claim 14, further comprising a reference system transformation module configured to determine transformation data based on an iterative closest point convergence method using the fine-granularity pose difference, and wherein the alignment module aligns the three dimensional point cloud map based on the transformation data.

16. The system of claim 12, wherein pose determination module determines the vehicle pose based on a two dimensional feature set from the sensor data, a three dimensional point cloud associated with the two dimensional feature set, and an image-plane re-projection error minimization method.

17. The system of claim 16, wherein the two dimensional feature set is associated with static features, and wherein the three dimensional point cloud is associated with static features.

18. The system of claim 12, wherein the pose determination module is configured to determine the vehicle pose based on a first two dimensional feature set from the sensor data, a second two dimensional feature set from the sensor data, a first three dimensional point cloud associated with the first two dimensional feature set, a second three dimensional point cloud associated with the second two dimensional feature set, and an image-plane re-projection error minimization method.

19. The system of claim 12, where the first two dimensional feature set is associated with static features, wherein the first three dimensional point cloud is associated with static features, wherein the second two dimensional feature set is associated with dynamic features, and wherein the second three dimensional point cloud is associated with dynamic features.

20. The system of claim 12, wherein the alignment module is configured to merge features of the aligned three dimensional map segments based on a confidence of the feature and a multi-partite matching method.