Occulsion aware planning and control

• US 10,642,275 B2
• Filed: 06/18/2018
• Issued: 05/05/2020
• Est. Priority Date: 06/18/2018
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Silva, William Anthony, Anguelov, Dragomir Dimitrov, Zwiebel, Benjamin Isaac, Kangaspunta, Juhana
• Primary Examiner(s): Kan, Yuri
• Application Number:
  US16/011,468
• Publication Number:
  US 20190384302A1
• Time in Patent Office: 687 Days
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4048 Field of view, e.g. obstructed view or direction of gaze
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 60/00272 relying on extrapolation of current movement
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object. An occlusion grid representing the occluded area can be stored as map data or can be dynamically generated. An occlusion grid can include occlusion fields, which represent discrete two- or three-dimensional areas of driveable environment. An occlusion field can indicate an occlusion state and an occupancy state, determined using LIDAR data and/or image data captured by the vehicle. An occupancy state of an occlusion field can be determined by ray casting LIDAR data or by projecting an occlusion field into segmented image data. The vehicle can be controlled to traverse the environment when a sufficient portion of the occlusion grid is visible and unoccupied.

Claims

1. A system comprising:

one or more processors that cause the system to perform operations comprising;

controlling an autonomous vehicle to traverse to a first location in an environment;

capturing, as sensor data, at least LIDAR data or image data of the environment using a sensor of the autonomous vehicle;

determining, based at least in part on accessing map data or on identifying an obstacle in the environment, an occlusion grid comprising a plurality of occlusion fields, an occlusion field of the occlusion grid indicating an occlusion state and an occupancy state of a corresponding region of the environment;

determining, as an occluded region and based at least in part on the sensor data, that at least a first portion of the occlusion grid is occluded at a first time;

determining, as an un-occluded region and based at least in part on the sensor data, that at least a second portion of the occlusion grid is un-occluded at the first time;

determining, based at least in part on an extent and occupancy of the second portion, a confidence level associated with safely traversing a trajectory through the environment;

controlling the autonomous vehicle to stay at the first location or to traverse to a second location based at least in part on the confidence level being below a threshold level;

determining, based at least in part on the sensor data, that a third portion of the occlusion grid is visible and unoccupied at a second time; and

controlling the autonomous vehicle based at least in part on the third portion of the occlusion grid being visible and unoccupied at the second time.

2. The system of claim 1, the operations further comprising one or more of:

determining that the autonomous vehicle is approaching an intersection where the autonomous vehicle is required to yield to other traffic;

determining that the obstacle is associated with a dynamic occlusion grid;

determining a predicted trajectory of a dynamic obstacle that traversed within the occlusion grid;

or

determining the occlusion grid based on a topographic occlusion region.

3. The system of claim 1, wherein the sensor data comprises the LIDAR data and the sensor is a LIDAR sensor, the operations further comprising:

ray casting a portion of the LIDAR data to determine the occlusion state and the occupancy state of the occlusion field.

4. The system of claim 1, wherein the sensor data is image data and the sensor is an image sensor, the operations further comprising:

performing segmentation on at least a portion of the image data to generate segmented image data identifying a driveable surface in the environment;

projecting the occlusion field into the segmented image data;

determining that the occlusion field projects into the driveable surface; and

controlling the autonomous vehicle to traverse the environment based at least in part on the occlusion field projecting into the driveable surface.

5. The system of claim 1, the operations further comprising:

determining a first number of un-occluded and unoccupied occlusion fields of the plurality of occlusion fields;

determining that the first number does not meet or exceed a threshold number of occlusion fields associated with safe traversal of the environment; and

controlling the autonomous vehicle to stay at the first location or to traverse to the second location based at least in part on the first number not meeting or exceeding the threshold number.

6. A method comprising:

controlling an autonomous vehicle to traverse to a first location in an environment;

determining, based at least in part on accessing map data or on identifying an obstacle in the environment, an occlusion grid;

determining, as an occluded region, that at least a first portion of the occlusion grid is occluded at a first time;

determining, as an un-occluded region, that at least a second portion of the occlusion grid is visible at the first time;

determining an occupancy state of the un-occluded region;

controlling the autonomous vehicle to stay at the first location or traverse to a second location based at least in part on an extent of the un-occluded region and the occupancy state of the un-occluded region;

determining that a third portion of the occlusion grid is un-occluded and unoccupied at a second time; and

controlling the autonomous vehicle based at least in part on the third portion of the occlusion grid being un-occluded and unoccupied at the second time.

7. The method of claim 6, further comprising:

determining that the autonomous vehicle is approaching an intersection in the environment, wherein the autonomous vehicle lacks a right of way at the intersection;

wherein;

the first location is a stop line associated with the intersection; and

the second location is a check line at least partially past the stop line and within the intersection.

8. The method of claim 7, further comprising:

controlling the autonomous vehicle to traverse from the first location to the second location at a first velocity below a threshold velocity;

receiving sensor data associated with the second location;

determining, based at least in part on the sensor data, that the third portion is un-occluded and unoccupied at the second time;

determining that an extent of the third portion meets or exceeds a threshold; and

controlling the autonomous vehicle to traverse from the second location through the intersection at a second velocity above the threshold velocity.

9. The method of claim 6, wherein the obstacle is a dynamic obstacle, the method further comprising;

identifying the dynamic obstacle in the environment;

associating the occlusion grid with the dynamic obstacle;

controlling the autonomous vehicle to stay at the first location based at least in part on the extent of the un-occluded region and the occupancy state of the un-occluded region at the first time; and

controlling the autonomous vehicle based at least in part on the third portion of the occlusion grid being un-occluded and unoccupied at the second time.

10. The method of claim 6, further comprising:

identifying the obstacle in the environment;

associating the occlusion grid with the obstacle;

identifying a dynamic object in the environment;

capturing motion data of the dynamic object in the environment;

determining that the dynamic object traversed into the occluded region;

determining a predicted trajectory for the dynamic object within the occluded region; and

controlling the autonomous vehicle based at least in part on the predicted trajectory of the dynamic object.

11. The method of claim 6, further comprising:

determining that the autonomous vehicle is within a threshold distance to a topographic occlusion region; and

determining, as a topographic occlusion grid, the occlusion grid based at least in part on the map data associated with the topographic occlusion region.

12. The method of claim 6, further comprising:

capturing LIDAR data using a LIDAR sensor on the autonomous vehicle; and

ray casting a portion of the LIDAR data to determine an occlusion state and an occupancy state of an occlusion field, the occlusion field representing a portion of the occlusion grid.

13. The method of claim 6, further comprising:

capturing image data using an image sensor on the autonomous vehicle;

performing segmentation on at least a portion of the image data to generate segmented image data identifying a driveable surface in the environment;

projecting an occlusion field into the segmented image data, the occlusion field representing a portion of the occlusion grid;

determining that the occlusion field projects into the driveable surface; and

controlling the autonomous vehicle to traverse the environment based at least in part on the occlusion field projecting into the driveable surface.

14. The method of claim 6, further comprising:

controlling the autonomous vehicle to traverse a portion of the environment based on a velocity selected based at least in part on a first extent of the portion to traverse in the environment and a second extent of the third portion of the occlusion grid being un-occluded and unoccupied.

15. The method of claim 6, further comprising:

determining a time period associated with the occluded region;

determining that no objects have entered the occluded region during the time period; and

determining, based at least in part on a speed limit associated with the occluded region, a confidence level associated with an occupancy of the occluded region based at least in part on the time period.

16. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising:

controlling a robotic platform to traverse to a first location in an environment;

determining a location of the robotic platform with respect to a map, the map comprising map data;

determining, based at least in part on accessing the map data associated with the location or on identifying an obstacle in the environment proximate to the robotic platform, an occlusion grid;

determining, as an occluded region, that at least a first portion of the occlusion grid is occluded at a first time;

determining, as an un-occluded region, that at least a second portion of the occlusion grid is visible at the first time;

determining an occupancy state of the un-occluded region;

controlling the robotic platform to stay at the first location or traverse to a second location based at least in part on an extent of the un-occluded region and the occupancy state of the un-occluded region;

determining that a third portion of the occlusion grid is un-occluded and unoccupied at a second time; and

controlling the robotic platform based at least in part on the third portion being un-occluded and unoccupied at the second time.

17. The non-transitory computer-readable medium of claim 16, the operations further comprising:

determining that the robotic platform is approaching an intersection in the environment, wherein the robotic platform is required to yield to other vehicles;

wherein;

the first location is a stop line associated with the intersection; and

the second location is a check line at least partially past the stop line and within the intersection.

18. The non-transitory computer-readable medium of claim 16, the operations further comprising:

capturing LIDAR data using a LIDAR sensor on the robotic platform;

ray casting a portion of the LIDAR data to determine an occupancy of an occlusion field, the occlusion field representing a portion of the occlusion grid; and

determining, based at least in part on the occupancy, that the occlusion grid is visible and unoccupied.

19. The non-transitory computer-readable medium of claim 16, the operations further comprising:

capturing image data using an image sensor on the robotic platform;

performing segmentation on at least a portion of the image data to generate segmented image data identifying a driveable surface in the environment;

projecting an occlusion field into the segmented image data, the occlusion field representing a portion of the occlusion grid;

determining that the occlusion field projects into the driveable surface; and

controlling the robotic platform to traverse the environment based at least in part on the occlusion field projecting into the driveable surface.

20. The non-transitory computer-readable medium of claim 16, the operations further comprising:

determining a time period associated with the occluded region;

determining that no objects have entered the occluded region during the time period; and

determining a confidence level associated with a safe trajectory.