Plane estimation for contextual awareness

• US 10,198,643 B1
• Filed: 12/19/2016
• Issued: 02/05/2019
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Ferguson, David I., Silver, David Harrison, Murveit, Hyman Jack
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Rivera-Martinez, Guillermo M
• Application Number:
  US15/383,317
• Time in Patent Office: 778 Days
• Field of Search: None
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

Aspects of the disclosure relate to classifying the status of objects. For examples, one or more computing devices detect an object from an image of a vehicle'"'"'s environment. The object is associated with a location. The one or more computing devices receive data corresponding to the surfaces of objects in the vehicle'"'"'s environment and identifying data within a region around the location of the object. The one or more computing devices also determine whether the data within the region corresponds to a planar surface extending away from an edge of the object. Based on this determination, the one or more computing devices classify the status of the object.

Claims

1. A method for determining whether a vehicle should respond to stop signs, the method comprising:

detecting, by one or more computing devices, an object and a location of the object from an image of the vehicle'"'"'s environment;

determining, by the one or more computing devices, that the object is a stop sign;

receiving, by the one or more computing devices, sensor data identifying locations and surfaces of objects within the vehicle'"'"'s environment including the stop sign;

determining, by the one or more computing devices, that the stop sign is a temporary stop sign by comparing the location to map information for the vehicle'"'"'s environment;

when the stop sign is determined to be temporary, processing, by the one or more computing devices, a portion of the sensor data with a predetermined volume of space around the location of the stop sign to determine whether a surface of the stop sign extends away from a second surface; and

determining, by the one or more computing devices, whether the vehicle should respond to the stop sign based on the determination of whether the surface of the stop sign extends away from the second surface.

2. The method of claim 1, further comprising, when the surface of the stop sign extends away from a surface of another object, maneuvering the vehicle, by the one or more computing devices, to ignore the stop sign.

3. The method of claim 1, further comprising, when the surface of the stop sign does not away from a surface of another object, maneuvering the vehicle, by the one or more computing devices, to respond to and obey the stop sign.

4. The method of claim 1, wherein processing the portion of the sensor data includes determining that the object is a reflection of a stop sign, the method further comprises ignoring the stop sign.

5. The method of claim 1, wherein processing the portion of the sensor data includes determining that the stop sign is located on a flat surface, the method further comprises ignoring the stop sign.

6. The method of claim 1, wherein processing the portion of the sensor data includes determining that there is no second surface, and the method further comprising, maneuvering the vehicle, by the one or more computing devices, to respond to and obey the stop sign.

7. The method of claim 1, wherein processing the portion of the sensor data includes determining a distance between the stop sign and the second surface, and wherein determining whether the vehicle should respond to the stop sign is further based on the distance.

8. The method of claim 1, wherein processing the portion of the sensor data includes determining a distance that the second surface extends away from the surface of the stop sign, and wherein determining whether the vehicle should respond to the stop sign is further based on the distance.

9. A system for determining whether a vehicle should respond to stop signs, the system comprising one or more computing devices configured to:

detecting an object and a location of the object from an image of the vehicle'"'"'s environment;

determining that the object is a stop sign;

receiving sensor data identifying locations and surfaces of objects within the vehicle'"'"'s environment including the stop sign;

determining that the stop sign is a temporary stop sign by comparing the location to map information for the vehicle'"'"'s environment;

when the stop sign is determined to be temporary, processing a portion of the sensor data with a predetermined volume of space around the location of the stop sign to determine whether a surface of the stop sign extends away from a second surface; and

determining whether the vehicle should respond to the stop sign based on the determination of whether the surface of the stop sign extends away from the second surface.

10. The system of claim 9, and the or more computing devices are further configured to, when the surface of the stop sign extends away from a surface of another object, maneuver the vehicle, by the one or more computing devices, to ignore the stop sign.

11. The system of claim 9, and the or more computing devices are further configured to, when the surface of the stop sign does not away from a surface of another object, maneuver the vehicle to respond to and obey the stop sign.

12. The system of claim 9, wherein processing the portion of the sensor data includes determining that the object is a reflection of a stop sign, the method further comprises ignoring the stop sign.

13. The system of claim 9, wherein processing the portion of the sensor data includes determining that the stop sign is located on a flat surface, and the or more computing devices are further configured to ignore the stop sign.

14. The system of claim 9, wherein processing the portion of the sensor data includes determining that there is no second surface, and the or more computing devices are further configured to maneuvering the vehicle to respond to and obey the stop sign.

15. The system of claim 9, wherein processing the portion of the sensor data includes determining a distance between the stop sign and the second surface, and wherein determining whether the vehicle should respond to the stop sign is further based on the distance.

16. The system of claim 9, wherein processing the portion of the sensor data includes determining a distance that the second surface extends away from the surface of the stop sign, and wherein determining whether the vehicle should respond to the stop sign is further based on the distance.

17. The system of claim 9, further comprising one or more sensors configured to generate the sensor data.

18. The system of claim 9, further comprising the vehicle.

19. A non-transitory recording medium on which instructions are stored, the instructions, when executed by one or more computing devices, cause the computing device to perform method for determining whether a vehicle should respond to stop signs, the method comprising:

detecting an object and a location of the object from an image of the vehicle'"'"'s environment;

determine that the object is a stop sign;

receive sensor data identifying locations and surfaces of objects within the vehicle'"'"'s environment including the stop sign;

determine that the stop sign is a temporary stop sign by comparing the location to map information for the vehicle'"'"'s environment;

when the stop sign is determined to be temporary, process a portion of the sensor data with a predetermined volume of space around the location of the stop sign to determine whether a surface of the stop sign extends away from a second surface; and

determine whether the vehicle should respond to the stop sign based on the determination of whether the surface of the stop sign extends away from the second surface.

20. The medium of claim 19, wherein the method further comprises, when the surface of the stop sign extends away from a surface of another object, maneuvering the vehicle to ignore the stop sign.