Using behavior of objects to infer changes in a driving environment

• US 9,600,768 B1
• Filed: 04/16/2013
• Issued: 03/21/2017
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I.
• Primary Examiner(s): Traktovenko, Ilya
• Application Number:
  US13/863,613
• Time in Patent Office: 1,435 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

An apparatus and method are disclosed for determining whether a driving environment has changed relative to a detailed map stored by an autonomous vehicle. An autonomous driving computer system of the autonomous vehicle may determine whether the driving environment has probably changed based on the location of one or more objects detected in the driving environment. The autonomous driving computer system may include various object models, each object model being associated with an object type, and where each object model defines one or more probability values that a given object type is expected (or not expected) to be found at a given location. By aggregating the various probability values resulting from the detection of objects in the driving environment, and then comparing the aggregated probability values with one or more probability threshold values, the autonomous driving computer system may predict or determine whether the driving environment has probably changed.

Claims

1. A system comprising:

a memory configured to store map information for a driving environment of a vehicle and a plurality of object models, each object model being associated with a particular type of vehicle and defining probabilities of not detecting a vehicle of the particular type at various map locations in the driving environment of an autonomous vehicle, wherein the plurality of object models include models associated with particular types of objects including—

a passenger vehicle object type; and

one or more processors in communication with the memory, the one or more processors configured to;

receiving the map information from a map provider;

maneuver the vehicle in the driving environment using the map information;

while maneuvering the vehicle, obtain sensor information corresponding to a detected object in the driving environment from one or more sensors of the autonomous vehicle;

determine characteristics of the detected object based on the obtained sensor information;

determine a location of the detected object based on the obtained sensor information;

select an object model from the plurality of object models associated with a particular type of vehicle corresponding to the determined characteristics of the detected object such that the selected object model is the object model associated with the passenger vehicles object type;

use the defined probabilities of the selected object model to determine a probability value defining a likelihood of a vehicle of the determined type of vehicle appearing at the determined location, wherein the location of the detected object corresponds to a shoulder of a particular highway in the map information, and the probability value further defines a likelihood of a vehicle of the determined type of vehicle appearing on a shoulder of a generic highway;

compare the probability value with a probability threshold value; and

identify that the driving environment has changed from the map information when the probability value is less than the probability threshold value.

2. The system of claim 1, wherein the selected object model is defined based on prior observations of objects of the particular object type associated with the selected object model appearing at one or more locations of the driving environment.

3. The system of claim 2, wherein the prior observations comprise maintaining a plurality of counts for the objects of the particular object type associated with the selected object model, wherein each count corresponds to a number of times objects of the particular object type associated with the selected object model and having a predetermined object characteristic previously appeared at the one or more locations.

4. The system of claim 1, wherein the one or more processors are further configured to:

determine a situation type from the obtained sensor information, the situation type identifying a driving situation encounterable by the autonomous vehicle corresponding to a vehicular accident; and

determine the probability value further based on the situation type.

5. The system of claim 4, wherein the one or more processors determine the situation type by:

determining one or more situational characteristics from the obtained sensor information, wherein the one or more situational characteristics comprise object types for objects involved in the situation and object density for the objects involved in the situation;

comparing the determined one or more situational characteristics with one or more situational characteristics of one or more situational object models stored in the memory, wherein each situational object model defines one or more characteristics expected to be found during a corresponding situation; and

selecting a given one of the situational object models having the determined situation type based on the comparison of the determined one or more situational characteristics with the characteristics of the selected situational object model.

6. The system of claim 1, wherein the one or more processors are further configured to:

determine one or more object characteristics for a plurality of detected objects based on obtained sensor information;

determine a plurality of locations for each of the plurality of detected objects;

determine a plurality of probability values, wherein;

each probability value of the plurality is associated with a corresponding object of the plurality of detected objects; and

the plurality of probability values are determined by referencing one or more of the plurality of object models with one or more locations of the plurality of locations; and

aggregate the plurality of probability values into an aggregated probability value;

wherein identifying whether the driving environment has changed is further based on the aggregated probability value being greater than or equal to the probability threshold value.

7. The system of claim 1, wherein the plurality of object models comprise at least one object model associated with a particular object type being a vehicle object type and at least one object model associated with a particular object type being a non-vehicle object type.

8. The system of claim 1, wherein when the driving environment is identified to have changed from the map information, the one or more processors are further configured to retrieve updated map information including changes to the driving environment from the map information.

9. A method comprising:

maneuvering, with one or more processors, a vehicle in a driving environment using map information for the driving environment of the vehicle;

the vehicle, wherein the detected object is a pedestrian;

determining, with the one or more processors, one or more object characteristics for the detected object based on the obtained sensor information;

determining, with the one or more processors, a location of the detected object based on the obtained sensor information;

selecting, with the one or more processors, an object model from a plurality of object models, wherein;

each object model of the plurality of object models is associated with a particular type of object and defining probabilities of not detecting an object of the particular type of object at various map locations in the driving environment of an autonomous vehicle, wherein the plurality of object models include models associated with particular types of objects including at least a pedestrian type of object, and

the selected object model is associated with a particular type of object corresponding to the one or more object characteristics for the detected object such that the selected object model is the object model associated with the pedestrian type of object;

determine a probability value defining a likelihood of an object of the type of the detected object appearing at the determined location using the defined probabilities of the selected object model, the location of the detected object corresponds to a centerline of a particular lane of the map information, and the probability value further defines a likelihood of a pedestrian appearing within a centerline of a lane;

comparing, with the one or more processors, the probability value with a probability threshold value; and

identifying, with the one or more processors, that the driving environment has changed from the map information when the probability value is less than the probability threshold value.

10. The method of claim 9, wherein the selected object model is defined based on prior observations of objects of the particular object type associated with the selected object model appearing at one or more locations of the driving environment.

11. The method of claim 10, wherein the prior observations comprise maintaining a plurality of counts for objects of the particular object type associated with the selected object model, wherein each count corresponds to a number of times the objects of the particular object type associated with the selected object model and with a predetermined object characteristic have previously appeared at the one or more locations.

12. The method of claim 9, further comprising:

determining a situation type from the obtained sensor information, the situation type identifying a driving situation encounterable by the autonomous vehicle; and

determining the probability value based on referencing the selected object model with the determined situation type.

13. The method of claim 12, wherein determining the situation type comprises:

determining one or more situational characteristics from the obtained sensor information, wherein the one or more situational characteristics comprise object types for objects involved in the situation and object density for the objects involved in the situation;

comparing the determined one or more situational characteristics with one or more situational characteristics of one or more situational object models stored in the memory, wherein each situational object model defines one or more characteristics expected to be found during a corresponding situation; and

selecting a given one of the situational object models having the determined situation type based on the comparison of the determined one or more situational characteristics with the characteristics of the selected situational object model.

14. The method of claim 9, further comprising:

determining one or more object characteristics for a plurality of detected objects based on obtained sensor information;

determining a plurality of locations for each of the plurality of detected objects;

determining a plurality of probability values, wherein;

each probability value of the plurality is associated with a corresponding object of the plurality of detected objects; and

the plurality of probability values are determined by referencing one or more of the plurality of object models with one or more locations of the plurality of locations; and

aggregating the plurality of probability values into an aggregated probability value;

wherein identifying whether the driving environment has changed is further based on the aggregated probability value being greater than or equal to the probability threshold value.

15. The method of claim 9, wherein the plurality of object models comprise at least one object model associated with a particular object type being a vehicle object type and at least one object model associated with a particular object type being a non-vehicle object type.

16. A method comprising:

maneuvering, with one or more processors, a vehicle in a driving environment using map information for the driving environment of the vehicle;

while maneuvering the vehicle, obtaining, with the one or more processors, sensor information corresponding to a detected object in the driving environment from one or more sensors of an autonomous vehicle, wherein the detected object is a pedestrian;

determining, with the one or more processors, one or more object characteristics for the detected object based on the obtained sensor information;

determining, with the one or more processors, a location of the detected object based on the obtained sensor information;

selecting, with the one or more processors, an object model from a plurality of object models, wherein;

each object model of the plurality of object models is associated with a particular type of object and defining probabilities of not detecting an object of the particular type of object at various map locations in the driving environment of an autonomous vehicle, wherein the plurality of object models include models associated with particular types of objects including a pedestrian type of object, and

the selected object model is associated with a particular type of object corresponding to the one or more object characteristics for the detected object such that the selected object model is the object model associated with the pedestrian type of object;

determine a probability value defining a likelihood of an object of the type of the detected object appearing at the determined location using the defined probabilities of the selected object model, the location of the detected object corresponds to a shoulder of a particular highway in the map information, and the probability value further defines a likelihood of a pedestrian appearing on a shoulder of a generic highway;

comparing, with the one or more processors, the probability value with a probability threshold value; and

identifying, with the one or more processors, that the driving environment has changed from the map information when the probability value is less than the probability threshold value.

17. The method of claim 16, wherein the selected object model is defined based on prior observations of objects of the particular object type associated with the selected object model appearing at one or more locations of the driving environment.

18. The method of claim 17, wherein the prior observations comprise maintaining a plurality of counts for objects of the particular object type associated with the selected object model, wherein each count corresponds to a number of times the objects of the particular object type associated with the selected object model and with a predetermined object characteristic have previously appeared at the one or more locations.

19. The method of claim 16, further comprising:

determining a situation type from the obtained sensor information, the situation type identifying a driving situation encounterable by the autonomous vehicle; and

determining the probability value based on referencing the selected object model with the determined situation type.

20. The method of claim 19, wherein determining the situation type comprises:

determining one or more situational characteristics from the obtained sensor information, wherein the one or more situational characteristics comprise object types for objects involved in the situation and object density for the objects involved in the situation;

comparing the determined one or more situational characteristics with one or more situational characteristics of one or more situational object models stored in the memory, wherein each situational object model defines one or more characteristics expected to be found during a corresponding situation; and

selecting a given one of the situational object models having the determined situation type based on the comparison of the determined one or more situational characteristics with the characteristics of the selected situational object model.