Object detection based on known structures of an environment of an autonomous vehicle

• US 9,026,303 B1
• Filed: 09/08/2014
• Issued: 05/05/2015
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Ogale, Abhijit
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US14/480,377
• Time in Patent Office: 239 Days
• Field of Search: 701/28
• US Class Current: 701/28
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An autonomous vehicle may be configured to detect objects based on known structures of an environment. The vehicle may be configured to obtain image data from a sensor and be configured to operate in an autonomous mode. The image data may include data indicative of a known structure in the environment. The vehicle may include a computer system. The computer system may determine, based on a first portion of the image data, information indicative of an appearance of the known structure. The computer system may determine, based on a second portion of the image data, information indicative of an appearance of an unknown object in the environment. The computer system may also compare the information indicative of the appearance of the known structure with the information indicative of the appearance of the unknown object and provide instructions to control the vehicle in the autonomous mode based on the comparison.

Claims

1. A method comprising:

operating, by a computer system, at least one sensor on a vehicle to obtain one or more images of a portion of an environment of the vehicle, wherein the vehicle is configured to operate in an autonomous mode, and wherein the portion of the environment includes a road on which the vehicle is traveling;

identifying a first portion of the one or more images corresponding to a first region of the road, wherein the first region of the road is unobstructed;

identifying a second portion of the one or more images corresponding to a second region of the road;

comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images to determine whether the second region of the road includes an obstruction on the road; and

providing, by the computer system, instructions to control the vehicle in the autonomous mode based on comparison.

2. The method of claim 1, wherein the first region of the road is a region of the road within a first distance ahead of the vehicle.

3. The method of claim 2, wherein the second region of the road is a region of the road further ahead of the vehicle than the first region of the road.

4. The method of claim 1, wherein comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images to determine whether the second region of the road includes an obstruction on the road comprises determining that the second region of the road includes an obstruction on the road.

5. The method of claim 4, wherein providing, by the computer system, instructions to control the vehicle in the autonomous mode based on the comparison comprises controlling the vehicle in the autonomous mode to avoid the obstruction.

6. The method of claim 1, wherein operating, by the computer system, at least one sensor on the vehicle to obtain the one or more images of the portion of the environment of the vehicle comprises:

determining a location of the road in the environment; and

causing the at least one sensor to obtain the one or more images based on the location.

7. The method of claim 1,

wherein the information indicative of the first portion of the one or more images comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the second portion of the one or more images comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

8. The method of claim 7, wherein comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images comprises:

comparing one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.

9. A vehicle comprising:

a sensor configured to obtain one or more images of a portion of an environment of the vehicle, wherein the portion of the environment includes a road on which the vehicle is traveling; and

a computer system, wherein the computer system is configured to;

identify a first portion of the one or more images corresponding to a first region of the road, wherein the first region of the road is unobstructed;

identify a second portion of the one or more images corresponding to a second region of the road;

compare information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images to determine whether the second region of the road includes an obstruction on the road; and

provide instructions to control the vehicle in an autonomous mode based on the comparison.

10. The vehicle of claim 9, wherein the first region of the road is a region of the road within a first distance ahead of the vehicle.

11. The vehicle of claim 10, wherein the second region of the road is a region of the road further ahead of the vehicle than the first region of the road.

12. The vehicle of claim 9, wherein, in response to a determination that the second region of the road includes an obstruction on the road based on the comparison, the computer system is configured to provide instructions to control the vehicle in the autonomous mode to avoid the obstruction.

13. The vehicle of claim 9,

wherein the information indicative of the first portion of the one or more images comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the second portion of the one or more images comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

14. The vehicle of claim 13, wherein the computer system is configured to:

compare one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.

15. A non-transitory computer readable medium having stored therein instructions that, when executed by a computer system in a vehicle configured to operate in an autonomous mode, cause the computer system to perform functions comprising:

operating at least one sensor on the vehicle to obtain one or more images of a portion of an environment of the vehicle, wherein the portion of the environment includes a road on which the vehicle is traveling;

identifying a first portion of the one or more images corresponding to a first region of the road, wherein the first region of the road is unobstructed;

identifying a second portion of the one or more images corresponding to a second region of the road;

comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images to determine whether the second region of the road includes an obstruction on the road; and

providing instructions to control the vehicle in the autonomous mode based on the comparison.

16. The non-transitory computer readable medium of claim 15, wherein the first region of the road is a region of the road within a first distance ahead of the vehicle.

17. The non-transitory computer readable medium of claim 16, wherein the second region of the road is a region of the road further ahead of the vehicle than the first region of the road.

18. The non-transitory computer readable medium of claim 15, wherein comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the one or more images to determine whether the second region of the road includes an obstruction on the road comprises determining that the second region of the road includes an obstruction on the road, and wherein providing instructions to control the vehicle in the autonomous mode based on the comparison comprises controlling the vehicle in the autonomous mode to avoid the obstruction.

19. The non-transitory computer readable medium of claim 15,

wherein the information indicative of the first portion of the one or more images comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the second portion of the one or more images comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

20. The non-transitory computer readable medium of claim 19 wherein comparing information indicative of the first portion of the one or more images with information indicative of the second portion of the image to determine whether the second region of the road includes an obstruction on the road comprises:

comparing one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.