Apparatus, method and program for generating occupancy grid map

• US 10,354,150 B2
• Filed: 09/08/2016
• Issued: 07/16/2019
• Est. Priority Date: 10/22/2015
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Yamazaki, Masaki, Tasaki, Tsuyoshi, Katsuki, Rie, Watanabe, Tomoki, Sekine, Masahiro, Nishiyama, Manabu
• Primary Examiner(s): Shin, Soo
• Application Number:
  US15/259,108
• Publication Number:
  US 20170116487A1
• Time in Patent Office: 1,041 Days
• Field of Search: None
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals

Abstract

According to one embodiment, an apparatus for generating an occupancy grid map constituted with a two-dimensional grid around a first moving body includes processing circuitry configured to acquire data of a distance from the first moving body to an obstacle lying in surroundings of the first moving body; acquire first information indicating the movement status of the first moving body at current time and determine an operation mode of the first moving body; set a range of interest in a grid of a resolution higher than a resolution of the grid of the occupancy grid map, from the operation mode; and align the occupancy grid map to the range of interest and generate the occupancy grid map in such a manner that the obstacle exists in the grid of the occupancy grid map and the grid of the range of interest with respect to a position corresponding to the distance data.

Claims

1. An apparatus for generating, in a pixel orthogonal coordinate system, an occupancy grid map constituted with a two-dimensional grid around a first moving body, wherein

the pixel orthogonal coordinate system is a two-dimensional orthogonal coordinate system which is composed of a p axis and a q axis and is set for the occupancy grid map in a two-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

the apparatus comprising processing circuitry configured to:

acquire a distance data at a current time from the first moving body to an obstacle in the camera coordinate system;

predict a position at a next time of the current time of the obstacle, from the distance data;

acquire first information indicating a movement status of the first moving body at current time;

determine an operation mode of the first moving body at the current time, from the first information;

obtain a movement range of the first moving body at the next time, from the operation mode;

set the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

align the occupancy grid map to the range of interest; and

generate the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.

2. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

calculate a motion of the first moving body at the current time; and

update the occupancy grid map and the range of interest at the current time to the occupancy grid map and the range of interest at the next time, based on the motion of the first moving body.

3. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

acquire environment information of a path of the first moving body; and

generate an environment object indicated by the environment information in the grid of the occupancy grid map and the grid of the range of interest, in addition to the obstacle represented by the distance data.

4. The apparatus according to claim 1, wherein the processing circuitry is further configured to:

predict a movement destination of a second moving body at the next time, the second moving body moving around the first moving body; and

generate the occupancy grid map in such a manner that the movement destination of the second moving body exists as the obstacle in the grid of the occupancy grid map and the grid of the range of interest in addition to the obstacle represented by the distance data.

5. The apparatus according to claim 1, wherein the determined operation mode is one of an operation mode of a stop state of the first moving body, an operation mode of a stop state for right turn, an operation mode of a stop state for left turn, an operation mode of forward straight, an operation mode of forward straight for right turn, an operation mode of forward straight for left turn, and an operation mode of backward straight.

6. The apparatus according to claim 5, wherein the processing circuitry is further configured to:

generate a path candidate, which is a candidate of a range of movement of the first moving body, using the operation mode; and

set the range of interest to include an environment object indicated by environment information in the path candidate.

7. The apparatus according to claim 5, wherein the processing circuitry is further configured to:

generate a path candidate, which is a candidate of a range of movement of the first moving body, using the operation mode; and

set the range of interest to include a movement destination in the path candidate.

8. A method for generating, in a pixel orthogonal coordinate system, an occupancy grid map constituted with a two-dimensional grid around a first moving body, wherein

the pixel orthogonal coordinate system is a two-dimensional orthogonal coordinate system which is composed of a p axis and a q axis and is set for the occupancy grid map in a two-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

the method comprising:

acquiring a distance data at a current time from the first moving body to an obstacle in the camera coordinate system;

predicting a position at a next time of the current time of the obstacle, from the distance data;

acquiring first information indicating a movement status of the first moving body at current time;

determining an operation mode of the first moving body at the current time, from the first information;

obtaining a movement range of the first moving body at the next time, from the operation mode;

setting the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

aligning the occupancy grid map to the range of interest; and

generating the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.

9. A program stored in a non-transitory computer readable medium, causing a computer to perform a method for generating, in a pixel orthogonal coordinate system, an occupancy grid map composed of a two-dimensional grid around a first moving body, wherein

the pixel orthogonal coordinate system is a two-dimensional orthogonal coordinate system which is composed of a p axis and a q axis and is set for the occupancy grid map in a two-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

acquiring a distance data at a current time from the first moving body to an obstacle in the camera coordinate system;

predicting a position at a next time of the current time of the obstacle, from the distance data;

acquiring first information indicating a movement status of the first moving body at current time;

determining an operation mode of the first moving body at the current time, from the first information;

obtaining a movement range of the first moving body at the next time, from the operation mode;

setting the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

aligning the occupancy grid map to the range of interest; and

generating the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.

10. An apparatus for generating an occupancy grid map around a first moving body in a pixel orthogonal coordinate system constituted with a three-dimensional grid for each predetermined time, wherein

the pixel orthogonal coordinate system is a three-dimensional orthogonal coordinate system which is composed of a p axis, a q axis, and an r axis and is set for the occupancy grid map in a three-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

the apparatus comprising processing circuitry configured to:

acquire a data distance at a current time from the first moving body to an obstacle, in the camera coordinate system;

predict a position at a next time of the current time of the obstacle from the distance data;

acquire first information indicating a movement status of the first moving body at current time;

determine an operation mode of the first moving body at the current time, from the first information;

obtain a movement range of the first moving body at the next time, from the operation mode;

set the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

align the occupancy grid map to the range of interest; and

generate the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.

11. A method for generating an occupancy grid map around a first moving body in a pixel orthogonal coordinate system constituted with a three-dimensional grid for each predetermined time, wherein

the pixel orthogonal coordinate system is a three-dimensional orthogonal coordinate system which is composed of a p axis, a q axis, and an r axis and is set for the occupancy grid map in a three-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

the method comprising:

acquiring a data distance at a current time from the first moving body to an obstacle, in the camera coordinate system;

predicting a position at a next time of the current time of the obstacle from the distance data;

acquiring first information indicating a movement status of the first moving body at current time;

determining an operation mode of the first moving body at the current time, from the first information;

setting the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

aligning the occupancy grid map to the range of interest; and

generating the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.

12. A program stored in a non-transitory computer readable medium, causing a computer to perform a method for generating an occupancy grid map around a first moving body in a pixel orthogonal coordinate system constituted with a three-dimensional grid for each predetermined time, wherein

the pixel orthogonal coordinate system is a three-dimensional orthogonal coordinate system which is composed of a p axis, a q axis, and an r axis and is set for the occupancy grid map in a three-dimensional plane for each time T, an origin of the pixel orthogonal coordinate system is set at the first moving body, and a direction of the p axis is set at a runway direction of the first moving body at time T, and

a camera coordinate system is a three-dimensional orthogonal coordinate system which is composed of a horizontal X axis, a vertical Y axis, and a depth-wise Z axis and has a resolution higher than a resolution of the pixel orthogonal coordinate system, a position of an origin of the camera coordinate system is set at the first moving body, and the camera coordinate system is set such that the Z axis coincides with the p axis of the pixel orthogonal coordinate system,

the method comprising:

acquiring a data distance at a current time from the first moving body to an obstacle, in the camera coordinate system;

predicting a position at a next time of the current time of the obstacle from the distance data;

acquiring first information indicating a movement status of the first moving body at current time;

determining an operation mode of the first moving body at the current time, from the first information;

setting the movement range at the next time and a range of interest including a predicted position of the obstacle at the next time, in the same resolution as the resolution of the camera coordinate system;

aligning the occupancy grid map to the range of interest; and

generating the occupancy grid map in such a manner that the obstacle exists in the grid corresponding to a distance data in the occupancy grid map at the next time and the grid corresponding to the distance data in the range of interest at the next time.