Control of Autonomous Vehicle Based on Environmental Object Classification Determined Using Phase Coherent LIDAR Data

• US 20190317219A1
• Filed: 10/29/2018
• Published: 10/17/2019
• Est. Priority Date: 04/11/2018
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated
• Original Assignee: Aurora Innovation, Inc.
• Inventors: Smith, Warren, Eade, Ethan, Anderson, Sterling J., Bagnell, James Andrew, Nabbe, Bartholomeus C., Urmson, Christopher Paul
• Granted Patent: US 11,550,061 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/402 Type
• CPC Class Codes: B60W 2554/4042 Longitudinal speed
• CPC Class Codes: B60W 2754/00 Output or target parameters relating to objects
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 60/00272 relying on extrapolation of current movement
• CPC Class Codes: G01S 17/34 using transmission of continuous, frequency-modulated waves while heterodyning the received signal, or a signal derived therefrom, with a locally-generated signal related to the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/228 Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G05D 1/247 using signals provided by artificial sources external to the vehicle, e.g. navigation beacons
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Determining classification(s) for object(s) in an environment of autonomous vehicle, and controlling the vehicle based on the determined classification(s). For example, autonomous steering, acceleration, and/or deceleration of the vehicle can be controlled based on determined pose(s) and/or classification(s) for objects in the environment. The control can be based on the pose(s) and/or classification(s) directly, and/or based on movement parameter(s), for the object(s), determined based on the pose(s) and/or classification(s). In many implementations, pose(s) and/or classification(s) of environmental object(s) are determined based on data from a phase coherent Light Detection and Ranging (LIDAR) component of the vehicle, such as a phase coherent LIDAR monopulse component and/or a frequency-modulated continuous wave (FMCW) LIDAR component.

Claims

1. A method, comprising:

receiving, from a phase coherent Light Detection and Ranging (LIDAR) monopulse component of a vehicle, a group of LIDAR data points collectively capturing a plurality of points in an area of an environment of the vehicle,

each of the LIDAR data points of the group indicating a corresponding range and a corresponding velocity for a corresponding one of the points in the environment, and each being generated based on a corresponding sensing event of the phase coherent LIDAR monopulse component,

wherein the corresponding sensing events of the phase coherent LIDAR monopulse component each comprise a first receiver sensing event at a first receiver of the phase coherent LIDAR monopulse component and a second receiver sensing event at a second receiver of the phase coherent LIDAR monopulse component;

determining that a subgroup, of the LIDAR data points of the group, corresponds to an object of a particular classification, the determining based on both;

the corresponding ranges for multiple of the LIDAR data points of the subgroup, and

the corresponding velocities for multiple of the LIDAR data points of the subgroup; and

adapting autonomous control of the vehicle based on determining that the subgroup corresponds to the object of the particular classification.

2. The method of claim 1, further comprising:

determining at least one instantaneous velocity of the object based on at least one of the corresponding velocities of the subgroup, wherein the at least one of the corresponding velocities of the subgroup is utilized in determining the at least one instantaneous velocity of the object based on the determining that the subgroup corresponds to the object;

wherein adapting autonomous control of the vehicle is further based on the at least one instantaneous velocity.

3. The method of claim 2, wherein determining the at least one instantaneous velocity based on the at least one of the corresponding velocities of the subgroup comprises determining the at least one instantaneous velocity based on a function of a plurality of the corresponding velocities of the subgroup.

4. The method of claim 2, wherein adapting autonomous control of the vehicle based on the at least one instantaneous velocity comprises:

determining at least one candidate trajectory of the object based on the at least one instantaneous velocity; and

adapting autonomous control of the vehicle based on the at least one candidate trajectory.

5. The method of claim 4, wherein determining the at least one candidate trajectory of the object based on the at least one instantaneous velocity comprises:

adapting a previously determined velocity, for the object, based on the at least one instantaneous velocity; and

determining the at least one candidate trajectory of the object based on the adaptation of the previously determined velocity.

6. The method of claim 2, wherein determining the at least one instantaneous velocity based on the at least one of the corresponding velocities of the subgroup comprises:

selecting an instantaneous velocity determination technique, from a plurality of candidate instantaneous velocity determination techniques, based on the instantaneous velocity determination technique being assigned to the particular classification; and

using the selected instantaneous velocity determination technique in determining the at least one instantaneous velocity.

7. The method of claim 2, wherein determining the at least one instantaneous velocity based on the at least one of the corresponding velocities of the subgroup comprises:

selecting an instantaneous velocity determination technique, from a plurality of candidate instantaneous velocity determination techniques, based on a quantity of the corresponding velocities of the subgroup; and

using the selected instantaneous velocity determination technique in determining the at least one instantaneous velocity.

8. The method of claim 1, wherein a corresponding angle is defined for each of the LIDAR data points of the group, and wherein the corresponding angles are each defined to a degree that is less than a beam-width of corresponding optical output, from a laser of the phase coherent LIDAR monopulse component, that resulted in the corresponding sensing event based on which the LIDAR data point is generated.

9. The method of claim 1, further comprising generating a corresponding angle for each of the LIDAR data points of the group, wherein generating a given corresponding angle, of the corresponding angles, for a given LIDAR data point of the LIDAR data points comprises:

generating the given corresponding angle based on comparing;

first data from the first receiver sensing event for the corresponding sensing event of the given LIDAR data point, and

second data from the second receiver sensing event for the corresponding sensing event of the given LIDAR data point.

10. The method of claim 9, wherein comparing the first data and the second data comprises comparing one or both of:

phase differences between the first data and the second data, and

amplitude differences between the first data and the second data.

11. The method of claim 10, wherein the first data and the second data are each a respective time varying intermediate frequency waveform, or a respective range-Doppler image.

12. The method of claim 1, wherein each of the LIDAR data points of the group is a corresponding super-resolution LIDAR data point constructed based on combining first data from the first receiver sensing event of the corresponding sensing event and second data from the second receiver sensing event of the corresponding sensing event.

13. The method of claim 1, wherein each of the LIDAR data points of the group further indicate a corresponding intensity, and wherein determining that the subgroup of the LIDAR data points of the group corresponds to the object of the particular classification is further based on the corresponding intensities for multiple of the LIDAR data points of the subgroup.

14. The method of claim 1, further comprising:

receiving, from the phase coherent LIDAR monopulse component immediately prior to receiving the group, a prior group of prior LIDAR data points that collectively capture a plurality of the points in the area of the environment of the vehicle,

each of the prior LIDAR data points of the prior group indicating a corresponding prior range and a corresponding prior velocity for the corresponding points in the environment, and each being generated based on a prior corresponding sensing event of the phase coherent LIDAR monopulse component;

wherein determining that the subgroup, of the LIDAR data points of the group, corresponds to the object of the particular classification is further based on both;

the corresponding prior ranges for multiple of the prior LIDAR data points, and

the corresponding prior velocities for multiple of the prior LIDAR data points.

15. The method of claim 1, wherein determining that the subgroup, of the LIDAR data points of the group, corresponds to the object of the particular classification comprises:

processing the LIDAR data points of the group using a trained machine learning model;

generating, based on processing of the LIDAR data points of the group using the trained machine learning model, an output that indicates that the subgroup has the particular classification; and

determining that the subgroup corresponds to the object having the particular classification based on the output indicating that the subgroup has the particular classification.

16. The method of claim 15,

wherein the output indicates, for each of a plurality of spatial regions of the LIDAR data points of the group, a corresponding probability that the corresponding spatial region has the particular classification, and

wherein determining that the subgroup corresponds to the object having the particular classification based on the output indicating that the subgroup has the particular classification comprises:

determining a set of one or more of the spatial regions based on the corresponding probabilities of the spatial regions of the set each satisfying a threshold; and

determining the subgroup based on the subgroup corresponding to the set of the one or more of the spatial regions.

17. The method of claim 16, wherein the spatial regions indicated by the output each correspond to either:

a corresponding one of the LIDAR data points of the group, or

a corresponding unique grouping of two or more of the LIDAR data points of the group.

18. The method of claim 16,

wherein the output indicates, for each of a plurality of spatial regions of the LIDAR data points of the group, a corresponding first probability that the corresponding spatial region has the particular classification and a corresponding additional probability that the corresponding spatial region has a distinct particular classification, and

wherein determining that the subgroup corresponds to the object having the particular classification based on the output indicating that the subgroup has the particular classification comprises:

determining a set of one or more of the spatial regions based on the corresponding probabilities of the spatial regions of the set each satisfying a threshold; and

determining the subgroup based on the subgroup corresponding to the set of the one or more of the spatial regions.

19. The method of claim 18, further comprising:

determining that an additional subgroup of the LIDAR data corresponds to an additional object having the distinct particular classification based on;

determining an additional set of one or more of the spatial regions based on the corresponding additional probabilities of the spatial regions of the additional set each satisfying a threshold; and

determining the additional subgroup based on the additional subgroup corresponding to the additional set of the one or more of the spatial regions;

wherein adapting autonomous control of the vehicle is further based on determining that the additional subgroup corresponds to the additional object having the distinct particular classification.

20. The method of claim 1, wherein adapting autonomous control of the vehicle based on determining that the subgroup corresponds to the object of the particular classification comprises altering at least one of:

a velocity of the vehicle, or

a direction of the vehicle.

21. The method of claim 17, wherein adapting autonomous control of the vehicle based on determining that the subgroup corresponds to the object of the particular classification comprises:

determining a pose of the object based on the LIDAR data points of the subgroup; and

adapting autonomous control based on the pose of the object and based on the particular classification of the object.

22. The method of claim 1, wherein adapting autonomous control of the vehicle based on determining that the subgroup corresponds to the object of the particular classification comprises:

determining that the object is mobile based on the particular classification;

determining a pose of the object based on the LIDAR data points of the subgroup;

determining, in response to determining that the object is mobile, at least one candidate trajectory for the object based on the pose of the object; and

adapting autonomous control of the vehicle based on the at least one candidate trajectory.

23. The method of claim 23, further comprising:

determining an instantaneous velocity of the object based on the LIDAR data points of the subgroup;

wherein determining the at least one candidate trajectory for the object is further based on the instantaneous velocity of the object.

24. An autonomous vehicle, comprising:

a phase coherent Light Detection and Ranging (LIDAR) monopulse component;

one or more processors executing stored computer instructions to;

receive, from the phase coherent LIDAR monopulse component, a group of phase coherent LIDAR data points collectively capturing a plurality of points in an area of an environment of the vehicle,

each of the phase coherent LIDAR data points of the group indicating a corresponding range and a corresponding velocity for a corresponding one of the points in the environment, and each being generated based on a corresponding sensing event of the phase coherent LIDAR monopulse component,

wherein the corresponding sensing events of the phase coherent LIDAR monopulse component each comprise a first receiver sensing event at a first receiver of the phase coherent LIDAR monopulse component and a second receiver sensing event at a second receiver of the phase coherent LIDAR monopulse component;

determine that a subgroup, of the phase coherent LIDAR data points of the group, corresponds to an object of a particular classification, the determining based on both;

the corresponding ranges for multiple of the phase coherent LIDAR data points of the subgroup, and

the corresponding velocities for multiple of the phase coherent LIDAR data points of the subgroup; and

control steering, acceleration, and/or deceleration of the vehicle based on determining that the subgroup corresponds to the object of the particular classification.

25. A method, comprising:

receiving, from a phase coherent Light Detection and Ranging (LIDAR) monopulse component of a vehicle, a group of LIDAR data points collectively capturing a plurality of points in an area of an environment of the vehicle,

each of the phase coherent LIDAR data points of the group indicating a corresponding range and a corresponding velocity for a corresponding one of the points in the environment, and each being generated based on a corresponding sensing event of the FMCW LIDAR monopulse component,

wherein the corresponding sensing events of the LIDAR monopulse component each comprise a first receiver sensing event at a first receiver of the phase coherent LIDAR monopulse component and a second receiver sensing event at a second receiver of the phase coherent LIDAR monopulse component;

determining that a subgroup, of the phase coherent LIDAR data points of the group, corresponds to an object of a particular classification, the determining based on both;

the corresponding ranges for multiple of the LIDAR data points of the subgroup, and

the corresponding velocities for multiple of the LIDAR data points of the subgroup;

determining a multi-dimensional pose of the object based on the subgroup of the LIDAR data points of the group; and

providing the determined multi-dimensional pose and an indication of the particular classification to a planning subsystem of the vehicle, wherein the planning subsystem of the vehicle generates a trajectory of the vehicle based on the multi-dimensional pose and the indication of the particular classification.