Methods and devices for determining movements of an object in an environment

• US 8,989,944 B1
• Filed: 11/26/2013
• Issued: 03/24/2015
• Est. Priority Date: 11/26/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Dolgov, Dmitri, Agarwal, Pratik
• Primary Examiner(s): Marc, McDieunel
• Application Number:
  US14/090,308
• Time in Patent Office: 483 Days
• Field of Search: 701/23, 701/2, 701/28, 340/425.5, 340/435, 340/576, 345/7
• US Class Current: 701/23
• CPC Class Codes: G01S 13/588 deriving the velocity value from the range measurement
• CPC Class Codes: G01S 13/589 measuring the velocity vector
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/588 measuring the velocity vector
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 in the back of the vehicles
• CPC Class Codes: G01S 2013/93273 on the top of the vehicles
• CPC Class Codes: G01S 2013/93275 in the bumper area
• CPC Class Codes: G01S 2013/93277 in the lights
• CPC Class Codes: G01S 2015/938 : in the bumper area radar in bumper area G01S2013/93275; bumper with obstacle sensor of electric or electronic type B60R19/483
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An example method may include receiving a first set of points based on detection of an environment of an autonomous vehicle during a first time period, selecting a plurality of points from the first set of points that form a first point cloud representing an object in the environment, receiving a second set of points based on detection of the environment during a second time period which is after the first period, selecting a plurality of points from the second set of points that form a second point cloud representing the object in the environment, determining a transformation between the selected points from the first set of points and the selected points from the second set of points, using the transformation to determine a velocity of the object, and providing instructions to control the autonomous vehicle based at least in part on the velocity of the object.

Claims

1. A computer-implemented method, comprising:

receiving a first set of points based on detection of an environment of an autonomous vehicle by one or more sensors on the autonomous vehicle during a first time period;

selecting a plurality of points from the first set of points that form a first point cloud representing an object in the environment;

receiving a second set of points based on detection of the environment by the one or more sensors on the autonomous vehicle during a second time period, wherein the second time period is after the first period;

selecting a plurality of points from the second set of points that form a second point cloud representing the object in the environment;

determining, by a control system of the autonomous vehicle, a transformation between the selected points from the first set of points that form the first point cloud representing the object in the environment and the selected points from the second set of points that form the second point cloud representing the object in the environment;

using the transformation to determine a velocity of the object; and

controlling the autonomous vehicle based at least in part on the velocity of the object.

2. The method of claim 1, wherein using the transformation to determine the velocity of the object comprises determining at least one of a position and a heading of the object.

3. The method of claim 1, further comprising before selecting the plurality of points from the first set of points, filtering the first set of points to remove one or more points generated based on detection of a ground surface in the environment.

4. The method of claim 1, wherein selecting the plurality of points from the first set of points that form the first point cloud representing the object comprises selecting points such that adjacent selected points have a pairwise distance that is less than a threshold distance.

5. The method of claim 1, wherein determining the transformation comprises determining a translation and a rotation between the selected points from the first set of points and the selected points from the second set of points so as to minimize a distance between selected points from the first set of points and associated points from the second set of points.

6. The method of claim 1, further comprising providing instructions to control the autonomous vehicle so as to avoid the object.

7. The method of claim 6, wherein providing instructions to control the autonomous vehicle so as to avoid the object comprises determining a plurality of contour outlines representing the object based on at least one of the first point cloud and the second point cloud.

8. The method of claim 1, further comprising:

receiving a third set of points based on detection of the environment during a third time period, wherein the third time period is after the second time period;

selecting a plurality of points from the third set of points that form a third point cloud representing the object in the environment;

determining a transformation between the selected points from the second set of points and the selected points from the third set of points; and

using the transformation between the selected points from the first set of points and the selected points from the second set of points and the transformation between the selected points from the second set of points and the selected points from the third set of points to determine an acceleration of the object.

9. The method of claim 1, further comprising:

selecting a plurality of points from the first set of points that form point cloud representations of one or more additional objects in the environment;

selecting a plurality of points from the second set of points that form point cloud representations of the one or more additional objects in the environment;

determining a transformation between the selected points from the first set of points and the selected from the second set of points for the one or more additional objects; and

using the transformation to determine a velocity of the one or more additional objects.

10. An autonomous vehicle, comprising:

one or more depth sensors; and

a control system configured to;

receive a first set of points based on detection of an environment by the one or more depth sensors on the autonomous vehicle during a first time period;

select a plurality of points from the first set of points that form a first point cloud representing an object in the environment;

receive a second set of points based on detection of the environment by the one or more depth sensors during a second time period, wherein the second time period is after the first time period;

select a plurality of points from the second set of points that form a second point cloud representing the object in the environment;

determine a transformation between the selected points from the first set of points that form the first point cloud representing the object in the environment and the selected from the second set of points that form the second point cloud representing the object in the environment;

use the transformation to determine a velocity of the object; and

control the autonomous vehicle based at least in part on the velocity of the object.

11. The vehicle of claim 10, wherein the control system is configured to determine the velocity of the object by determining at least one of a position and a heading of the object.

12. The vehicle of claim 10, wherein the control system is further configured to before selecting the plurality of points from the first set of points, filter the first set of points to remove one or more points generated based on detection of a ground surface in the environment.

13. The vehicle of claim 10, wherein the control system is further configured to provide instructions to control the vehicle so as to avoid the object.

14. The vehicle of claim 10, wherein the control system is further configured to:

receive the first set of points in the environment from one of a plurality of depth sensors on the vehicle; and

provide instructions to control the vehicle so as to avoid the object based on sensor data from any one of the plurality of depth sensors.

15. The vehicle of claim 10, wherein the control system is further configured to:

receive a third set of points based on detection of the environment during a third time period by the depth sensor, wherein the third time period is after the second time period;

select a plurality of points from the third set of points that form a third point cloud representing the object in the environment;

determine a transformation between the selected points from the second set of points and the selected from the third set of points; and

use the transformation between the selected points from the first set of points and the selected points from the second set of points and the transformation between the selected points from the second set of points and the selected points from the third set of points to determine an acceleration of the object.

16. The vehicle of claim 10, wherein the control system is further configured to:

select a plurality of points from the first set of points that form point cloud representations of one or more additional objects in the environment;

select a plurality of points from the second set of points that form point cloud representations of the one or more additional objects in the environment;

determine a transformation between the selected points from the first set of points and the selected points from the second set of points for the one or more additional objects; and

use the transformation to determine a velocity of the one or more additional objects.

17. A non-transitory computer readable medium having stored therein instructions, that when executed by a control system of an autonomous vehicle, cause the control system to perform functions comprising:

receiving a first set of points based on detection of an environment of the autonomous vehicle by one or more sensors on the autonomous vehicle during a first time period;

selecting a plurality of points from the first set of points that form a first point cloud representing an object in the environment;

receiving a second set of points based on detection of the environment by the one or more sensors on the autonomous vehicle during a second time period, wherein the second time period is after the first period;

selecting a plurality of points from the second set of points that form a second point cloud representing the object in the environment;

determining a transformation between the selected points from the first set of points that form the first point cloud representing the object in the environment and the selected points from the second set of points that form the second point cloud representing the object in the environment;

using the transformation to determine a velocity of the object; and

controlling the autonomous vehicle based at least in part on the velocity of the object.

18. The non-transitory computer readable medium of claim 17, wherein using the transformation to determine the velocity of the object comprises determining at least one of a position and a heading of the object.

19. The non-transitory computer readable medium of claim 17, further comprising providing instructions, that when executed by the computing system, cause the computing system to perform a function comprising:

controlling the autonomous vehicle so as to avoid the object.