Occluded obstacle classification for vehicles

• US 10,137,890 B2
• Filed: 06/28/2016
• Issued: 11/27/2018
• Est. Priority Date: 06/28/2016
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Sakai, Katsuhiro, Mei, Xue, Kamata, Nobuhide
• Primary Examiner(s): Weber, Tamara L
• Application Number:
  US15/195,760
• Publication Number:
  US 20170369051A1
• Time in Patent Office: 882 Days
• Field of Search: None
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/60 Traversable objects, e.g. speed bumps or curbs
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 2720/00 Output or target parameters relating to overall vehicle dynamics
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/18 Propelling the vehicle
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 : for mapping or imaging
• CPC Class Codes: G01S 17/931 : of land vehicles
• CPC Class Codes: G01S 7/003 : Transmission of data between radar, sonar or lidar systems and remote stations
• CPC Class Codes: G01S 7/4802 : using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G06F 18/22 : Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/255 : Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/75 : Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Obstacles located in an external environment of a vehicle can be classified. At least a portion of the external environment can be sensed using one or more sensors to acquire sensor data. An obstacle candidate can be identified based on the acquired sensor data. An occlusion status for the identified obstacle candidate can be determined. The occlusion status can be a ratio of acquired sensor data for the obstacle candidate that is occluded to all acquired sensor data for the obstacle candidate. A classification for the obstacle candidate can be determined based on the determined occlusion status. A driving maneuver for the vehicle can be determined at least partially based on the determined classification for the obstacle candidate. The vehicle can be caused to implement the determined driving maneuver. The vehicle can be an autonomous vehicle.

Claims

1. An obstacle classification method for a vehicle, the method comprising:

acquiring, using one or more sensors, sensor data of at least a portion of an external environment of the vehicle;

identifying, using a processor, an obstacle candidate based on the acquired sensor data;

determining an occlusion status for the identified obstacle candidate, the occlusion status being a ratio of acquired sensor data for the obstacle candidate that is occluded to all acquired sensor data for the obstacle candidate, the acquired sensor data for the obstacle candidate that is occluded being defined by any acquired sensor data points for the obstacle candidate that are substantially adjacent to acquired sensor data points for one or more other obstacle candidates;

determining a classification for the obstacle candidate based on the determined occlusion status;

determining a driving maneuver for the vehicle at least partially based on the determined classification for the obstacle candidate; and

causing the vehicle to implement the determined driving maneuver.

2. The method of claim 1, wherein determining a classification for the obstacle candidate based on the determined occlusion status includes:

comparing the determined occlusion status to a first occlusion status threshold; and

if the determined occlusion status is above the first occlusion status threshold, determining the classification for the obstacle candidate as being indeterminate.

3. The method of claim 2, wherein, if the classification for the obstacle candidate is determined as being indeterminate, the driving maneuver is a conservative driving maneuver.

4. The method of claim 1, wherein the one or more sensors are one or more LIDAR sensors, and wherein the acquired sensor data is a plurality of object data points.

5. The method of claim 1, further including determining whether the acquired sensor data for the obstacle candidate is occluded.

6. The method of claim 5, wherein determining whether the acquired sensor data for the obstacle candidate is occluded is performed using ray tracing.

7. An obstacle classification method for a vehicle, the method comprising:

acquiring, using one or more sensors, sensor data of at least a portion of an external environment of the vehicle;

identifying, using a processor, an obstacle candidate based on the acquired sensor data;

determining an occlusion status for the identified obstacle candidate, the occlusion status being a ratio of acquired sensor data for the obstacle candidate that is occluded to all acquired sensor data for the obstacle candidate;

determining a classification for the obstacle candidate based on the determined occlusion status, the determining including;

comparing the determined occlusion status to a first occlusion status threshold;

if the determined occlusion status is above the first occlusion status threshold, determining the classification for the obstacle candidate as being indeterminate;

if the determined occlusion status is below the first occlusion status threshold, comparing the determined occlusion status to a second occlusion status threshold, the second occlusion status threshold being less than the first occlusion status threshold; and

if the determined occlusion status is above the second occlusion status threshold, determining the a classification for the obstacle candidate using one or more partially occluded obstacle models;

determining a driving maneuver for the vehicle at least partially based on the determined classification for the obstacle candidate; and

causing the vehicle to implement the determined driving maneuver.

8. The method of claim 7, wherein, if the obstacle candidate matches a partially occluded obstacle model, determining the classification for the obstacle candidate is based on a predetermined classification associated with the partially occluded obstacle model that the obstacle candidate matches.

9. The method of claim 7, further including:

if the determined occlusion status is below the second occlusion status threshold, determining the classification for the obstacle candidate using one or more full visibility obstacle models.

10. The method of claim 7, wherein the acquired sensor data for the obstacle candidate that is occluded includes at least all border sensor data for the obstacle candidate.

11. An obstacle classification system for a vehicle, the system comprising:

a sensor system including one or more sensors, the one or more sensors being configured to acquire sensor data of at least a portion of an external environment of the vehicle;

one or more actuators; and

a processor operatively connected to the sensor system, the processor operatively connected to the one or more actuators, the processor being programmed to initiate executable operations comprising;

identifying an obstacle candidate based on the acquired sensor data;

determining an occlusion status for the identified obstacle candidate, the occlusion status being a ratio of acquired sensor data for the obstacle candidate that is occluded to all acquired sensor data for the obstacle candidate, the acquired sensor data for the obstacle candidate that is occluded being defined by any acquired sensor data points for the obstacle candidate that are substantially adjacent to acquired sensor data points for one or more other obstacle candidates;

determining a classification for the obstacle candidate based on the determined occlusion status;

determining a driving maneuver for the vehicle at least partially based on the determined classification for the obstacle candidate; and

activating one or more actuators to cause the vehicle to implement the determined driving maneuver.

12. The system of claim 11, wherein determining a classification for the obstacle candidate based on the determined occlusion status includes:

comparing the determined occlusion status to a first occlusion status threshold; and

if the determined occlusion status is above the first occlusion status threshold, determining the classification for the obstacle candidate as being indeterminate.

13. The system of claim 12, wherein, if the classification for the obstacle candidate is determined as being indeterminate, the driving maneuver is a conservative driving maneuver.

14. The system of claim 11, wherein the one or more sensors are one or more LIDAR sensors, and wherein the acquired sensor data is a plurality of object data points.

15. The system of claim 11, wherein the executable operations further include:

determining whether the acquired sensor data for the obstacle candidate is occluded.

16. The system of claim 15, wherein determining whether the acquired sensor data for the obstacle candidate is occluded is performed using ray tracing.

17. An obstacle classification system for a vehicle, the system comprising:

a sensor system including one or more sensors, the one or more sensors being configured to acquire sensor data of at least a portion of an external environment of the vehicle;

one or more actuators; and

a processor operatively connected to the sensor system, the processor operatively connected to the one or more actuators, the processor being programmed to initiate executable operations comprising;

identifying an obstacle candidate based on the acquired sensor data;

determining an occlusion status for the identified obstacle candidate, the occlusion status being a ratio of acquired sensor data for the obstacle candidate that is occluded to all acquired sensor data for the obstacle candidate;

determining a classification for the obstacle candidate based on the determined occlusion status, the determining including;

comparing the determined occlusion status to a first occlusion status threshold;

if the determined occlusion status is above the first occlusion status threshold, determining the classification for the obstacle candidate as being indeterminate;

if the determined occlusion status is below the first occlusion status threshold, comparing the determined occlusion status to a second occlusion status threshold, the second occlusion status threshold being less than the first occlusion status threshold; and

if the determined occlusion status is above the second occlusion status threshold, determining the classification for the obstacle candidate using one or more partially occluded obstacle models;

determining a driving maneuver for the vehicle at least partially based on the determined classification for the obstacle candidate; and

activating one or more actuators to cause the vehicle to implement the determined driving maneuver.

18. The system of claim 17, wherein the executable operations further include:

if the obstacle candidate matches a partially occluded obstacle model, determining the classification for the obstacle candidate is based on a predetermined classification associated with the partially occluded obstacle model that the obstacle candidate matches.

19. The system of claim 17, wherein the executable operations further include:

if the determined occlusion status is below the second occlusion status threshold, determining the classification for the obstacle candidate using one or more full visibility obstacle models.

20. The system of claim 17, wherein the acquired sensor data for the obstacle candidate that is occluded includes at least all border sensor data for the obstacle candidate.