System and method for precision localization and mapping

• US 10,209,081 B2
• Filed: 08/09/2017
• Issued: 02/19/2019
• Est. Priority Date: 08/09/2016
• Status: Active Grant

• Current Assignee: Nauto, Inc.
• Original Assignee: Nauto, Inc.
• Inventors: Ghadiok, Vaibhav, Gleeson-May, Michael
• Primary Examiner(s): Shudy, Angelina
• Application Number:
  US15/673,098
• Publication Number:
  US 20180045519A1
• Time in Patent Office: 559 Days
• Field of Search: 701445
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3476 using point of interest [POI] information, e.g. a route passing visible POIs
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/582 : of traffic signs
• CPC Class Codes: G06V 2201/07 : Target detection

Abstract

A method for localization and mapping, including recording an image at a camera mounted to a vehicle, the vehicle associated with a global system location; identifying a landmark depicted in the image with a landmark identification module of a computing system associated with the vehicle, the identified landmark having a landmark geographic location and a known parameter; extracting a set of landmark parameters from the image with a feature extraction module of the computing system; determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the extracted set of landmark parameters and the known parameter; and updating, at the computing system, the global system location based on the relative position.

Claims

1. A method for vehicular localization and mapping, comprising:

at a camera mounted to a vehicle and oriented to define a field of view including a region forward of the vehicle during vehicle motion, wherein the vehicle is associated with a global system location, wherein the global system location is associated with a location error, recording a sequence of images of the region forward of the vehicle in response to the location error exceeding a threshold error value;

identifying a landmark depicted in an image of the sequence of images with a landmark identification module of a computing system associated with the vehicle, the landmark having a landmark geographic location and a known parameter, wherein the known parameter characterizes a physical dimension of the landmark;

identifying the landmark in each of the sequence of images;

extracting a set of landmark parameters from the image with a feature extraction module of the computing system, wherein the set of landmark parameters comprises a landmark pose;

extracting a corresponding landmark pose from each of the sequence of images;

determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the set of landmark parameters and the known parameter;

determining a temporal pattern based on the corresponding landmark pose extracted from each of the sequence of images;

determining a vehicle trajectory based on the temporal pattern, wherein the vehicle is controlled along the vehicle trajectory based on the location error, and wherein the vehicle is an autonomous vehicle;

updating, at the computing system, the global system location based on the relative position;

setting the location error to an error value below the threshold error value in response to updating the global system location; and

incrementing the location error in proportion to an elapsed time subsequent to updating the global system location.

2. The method of claim 1, further comprising retrieving the landmark geographic location of the landmark from a remote database associated with the computing system.

3. The method of claim 2, further comprising determining a set of unit regions based on the global system location, and retrieving the landmark geographic location from the remote database based on the set of unit regions.

4. The method of claim 2, wherein the set of landmark parameters comprises a textual location indicator, and further comprising determining a set of unit regions based on the textual location indicator, and retrieving the landmark geographic location from the remote database based on the set of unit regions.

5. The method of claim 1, further comprising determining a route based on an accumulation rate of the location error exceeding a threshold accumulation rate value, wherein the route is determined according to a landmark density along the route, and navigating the vehicle according to the route.

6. The method of claim 1, further comprising determining, at the computing system, a speed value and a heading value associated with the vehicle based on the set of landmark parameters, wherein the speed value is associated with a speed error and the heading value is associated with a heading error.

7. The method of claim 6, further comprising continuously updating the global system location based on the speed value and heading value, and continuously updating the location error based on the speed error and the heading error.

8. The method of claim 6, further comprising measuring a wheel rotations-per-minute value, determining the speed value based on the wheel rotations-per-minute value in combination with a known wheel diameter, and determining the heading value based on the set of landmark parameters.

9. The method of claim 1, wherein the landmark comprises a street sign having a set of corners, and wherein extracting the set of features comprises extracting a set of pixel positions of the set of corners from the image and determining the landmark pose based on the set of pixel positions.

10. The method of claim 1, further comprising:

identifying a plurality of landmarks depicted in the image, each of the plurality of landmarks associated with a corresponding geographic location;

extracting a set of landmark parameters associated with each of the plurality of landmarks from the image;

determining a static pattern based on the set of landmark parameters associated with each of the plurality of landmarks; and

determining the global system location based on the static pattern.