Apparatuses and methods for self-position calculation of a vehicle using a light projector and a camera

• US 10,026,196 B2
• Filed: 02/24/2014
• Issued: 07/17/2018
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Yamaguchi, Ichiro, Nishiuchi, Hidekazu
• Primary Examiner(s): Osifade, Idowu O
• Application Number:
  US15/118,240
• Publication Number:
  US 20170186186A1
• Time in Patent Office: 1,604 Days
• Field of Search: None
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A self-position calculating apparatus includes: a light projector projecting a patterned light beam onto a road surface around a vehicle; and a camera capturing an image of the road surface including an area onto which the patterned light beam is projected. The apparatus calculates an orientation angle of the vehicle from a position of the patterned light beam, and calculate an amount of change in orientation based on temporal changes in multiple feature points on the road surface. The apparatus calculates current position and orientation angle by adding the amount of change to initial position and orientation angle. If a condition under which the multiple feature points are detected does not satisfy a first criterion, the apparatus sets the current position at that time and the orientation angle at the initial position and orientation, and starts adding the amount of change to the initial position and orientation angle.

Claims

1. A self-position calculating apparatus comprising:

a light projector configured to project a patterned light beam onto a road surface around a vehicle;

a camera installed in the vehicle, and configured to capture an image of the road surface around the vehicle including an area onto which the patterned light beam is projected;

a processor; and

a memory that stores executable instructions, which, when executed by the processor, become operational with the processor to;

calculate an orientation angle of the vehicle relative to the road surface from a position of the patterned light beam on the image obtained by the camera;

calculate an amount of change in orientation of the vehicle based on temporal changes in a plurality of feature points on the road surface which are detected from the image obtained by the camera;

calculate current position and orientation angle of the vehicle by adding the amount of change in the orientation to initial position and orientation angle of the vehicle; and

determine under what condition the plurality of feature points are detected, wherein

if the condition under which the plurality of feature points are detected does not satisfy a first criterion, set the current position of the vehicle at that time and the orientation angle of the vehicle is calculated at the initial position and orientation angle of the vehicle and the amount of change in the orientation is added to the initial position and orientation angle.

2. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

if the number of the plurality of feature points used by the orientation change amount calculator is equal to or less than a predetermined threshold value, determine that the condition under which the plurality of feature points are detected does not satisfy the first criterion.

3. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

if density of the feature points detected from the image is equal to or less than predetermined threshold density, determine that the condition under which the plurality of feature points are detected does not satisfy the first criterion.

4. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

calculate a plane including the plurality of feature points from relative positions of the plurality of feature points detected from the image, wherein

if accuracy with which the plane is calculated is equal to or less than predetermined threshold accuracy, determine that the condition under which the plurality of feature points are detected does not satisfy the first criterion.

5. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

detect an amount of change in an illuminance around the vehicle, wherein

if the amount of change in the illuminance around the vehicle is equal to or greater than a predetermined threshold illuminance, determine that the condition under which the plurality of feature points are detected does not satisfy the first criterion.

6. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

determine a state of calculation of the orientation angle by the orientation angle calculator, wherein

if the condition under which the plurality of feature points are detected does not satisfy the first criterion, and if the state of the calculation of the orientation angle does not satisfy a second criterion, start to add the amount of change in the orientation to the initial orientation angle which is set with at least occupants and payload of the vehicle taken into consideration.

7. The self-position calculating apparatus according to claim 1, wherein the instructions stored in the memory further comprise instructions to:

determine a state of calculation of the orientation angle, wherein

if the condition under which the plurality of feature points are detected does not satisfy the first criterion, and if the state of the calculation of the orientation angle does not satisfy a second criterion, start to add the amount of change in the orientation to an orientation angle which is calculated before determining that the condition under which the plurality of feature points are detected does not satisfy the first criterion.

8. A self-position calculating method comprising:

a step of projecting a patterned light beam onto a road surface around a vehicle, by a light projector installed in the vehicle;

a step of capturing an image of the road surface around the vehicle including an area onto which the patterned light beam is projected, by a camera installed in the vehicle;

a step of calculating an orientation angle of the vehicle relative to the road surface from a position of the patterned light beam on the image, by a controller of the vehicle;

an orientation angle change amount calculating step of calculating an amount of change in the orientation of the vehicle based on temporal changes in a plurality of feature points on the road surface which are detected from the image, by the controller;

a self-position calculating step of calculating current position and orientation angle of the vehicle by adding the amount of change in the orientation to initial position and orientation angle of the vehicle, by the controller; and

a step of determining under what condition the plurality of feature points are detected in the orientation change amount calculating step, by the controller, wherein

if the condition under which the plurality of feature points are detected does not satisfy a first criterion, the self-position calculating step sets the current position of the vehicle at that time and the orientation angle of the vehicle calculated from the position of the patterned light beam at the initial position and orientation angle of the vehicle, and starts to add the amount of change in the orientation to the initial position and orientation angle.

9. A self-position calculating apparatus comprising:

a light projector configured to project a patterned light beam onto a road surface around a vehicle;

a camera installed in the vehicle and configured to capture an image of the road surface around the vehicle including an area onto which the patterned light beam is projected;

a processor; and

a memory that stores executable instructions, which, when executed by the processor, become operational with the processor to;

calculate an initial orientation angle of the vehicle relative to the road surface from a position of the patterned light beam on the image obtained by the camera;

calculate an amount of change in orientation of the vehicle based on temporal changes in feature points on the road surface that are detected from the image obtained by the camera;

update a current position and a current orientation angle of the vehicle by adding the amount of change in the orientation to an initial position and the initial orientation angle of the vehicle; and

based on a determination that a condition under which the feature points are detected does not satisfy a first criterion, set the current position and the current orientation angle of the vehicle as the initial position and the initial orientation angle of the vehicle.