Object detection based on known structures of an environment of an autonomous vehicle

• US 8,855,849 B1
• Filed: 02/25/2013
• Issued: 10/07/2014
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Ogale, Abhijit
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US13/775,554
• Time in Patent Office: 589 Days
• Field of Search: 701/28
• US Class Current: 701/28
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An autonomous vehicle may be configured to detect objects based on known structures of an environment. The vehicle may be configured to obtain image data from a sensor and be configured to operate in an autonomous mode. The image data may include data indicative of a known structure in the environment. The vehicle may include a computer system. The computer system may determine, based on a first portion of the image data, information indicative of an appearance of the known structure. The computer system may determine, based on a second portion of the image data, information indicative of an appearance of an unknown object in the environment. The computer system may also compare the information indicative of the appearance of the known structure with the information indicative of the appearance of the unknown object and provide instructions to control the vehicle in the autonomous mode based on the comparison.

Claims

1. A method comprising:

obtaining, by a computer system, image data from at least one sensor of a vehicle configured to operate in an autonomous mode in an environment, wherein the at least one sensor is coupled to the vehicle, and wherein the image data includes data indicative of a known structure on which the vehicle is traveling;

determining, based on a first portion of the image data, information indicative of an appearance of the known structure in the environment;

determining, based on a second portion of the image data, information indicative of an appearance of an unknown object in the environment, wherein the second portion of the image data is different than the first portion of the image data;

comparing the information indicative of the appearance of the known structure in the environment with the information indicative of the appearance of the unknown object in the environment so as to determine whether the unknown object is part of the known structure; and

providing, by the computer system, instructions to control the vehicle in the autonomous mode based on the comparison.

2. The method of claim 1, wherein the known structure comprises a road or a bridge.

3. The method of claim 1, wherein obtaining the image data from the at least one sensor of the vehicle comprises:

determining a location of the known structure in the environment; and

causing the at least one sensor to obtain the image data based on the location.

4. The method of claim 1,

wherein the information indicative of the appearance of the known structure comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the appearance of the unknown structure comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

5. The method of claim 4, wherein comparing the information indicative of the appearance of the known structure in the environment with the information indicative of the appearance of the unknown object in the environment so as to determine whether the unknown object is part of the known structure comprises:

comparing one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.

6. The method of claim 5, further comprising:

obtaining second image data, wherein the second image data is different than the first image data;

determining, based on the second image data, information indicative of an appearance of a second unknown object in the environment, wherein the information indicative of the second unknown object comprises one or more of a third color average, a third color-intensity, a third shadow presence, a third variation of color-intensity, or a third variation of color; and

comparing the information indicative of the appearance of the second unknown object in the environment with the information indicative of the appearance of the known structure so as to determine whether the second unknown object is part of the known structure.

7. The method of claim 6, wherein comparing the information indicative of the appearance of the second unknown object in the environment with the information indicative of the appearance of the known structure so as to determine whether the second unknown object part of the known structure comprises:

comparing one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the third color average, the third color-intensity, the third shadow presence, the third variation of color-intensity, or the third variation of color.

8. A vehicle comprising:

a sensor configured to obtain image data in an environment, wherein the image data includes data indicative of a known structure on which the vehicle is traveling, and wherein the vehicle is configured to operate in an autonomous mode in the environment; and

a computer system, wherein the computer system is configured to;

determine, based on a first portion of the image data, information indicative of an appearance of the known structure in the environment;

determine, based on a second portion of the image data, information indicative of an appearance of an unknown object in the environment, wherein the second portion of the image data is different than the first portion of the image data;

compare the information indicative of the appearance of the known structure in the environment with the information indicative of the appearance of the unknown object in the environment so as to determine whether the unknown object is part of the known structure; and

provide instructions to control the vehicle in the autonomous mode based on the comparison.

9. The vehicle of claim 8, wherein the known structure comprises a road or a bridge.

10. The vehicle of claim 8, wherein the computer system is further configured to:

determine a location of the known structure in the environment; and

cause the sensor to obtain the image data based on the location.

11. The vehicle of claim 8,

wherein the information indicative of the appearance of the known structure comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the appearance of the unknown structure comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

12. The vehicle of claim 11, wherein the computer system is configured to:

compare one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.

13. The vehicle of claim 12, wherein the computer system is further configured to:

obtain second image data, wherein the second image data is different than the first image data;

determine, based on the second image data, information indicative of an appearance of a second unknown object in the environment, wherein the information indicative of the second unknown object comprises one or more of a third color average, a third color-intensity, a third shadow presence, a third variation of color-intensity, or a third variation of color; and

compare the information indicative of the appearance of second unknown object with the information indicative of the appearance of the known structure so as to determine whether the second unknown object is part of the known structure.

14. The vehicle of claim 13, wherein the computer system is further configured to compare one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the third color average, the third color-intensity, the third shadow presence, the third variation of color-intensity, or the third variation of color.

15. A non-transitory computer readable medium having stored therein instructions that when executed by a computer system in a vehicle, cause the computer system to perform functions comprising:

obtaining image data from at least one sensor of the vehicle that is configured to operate in an autonomous mode in an environment, wherein the at least one sensor is coupled to the vehicle, and wherein the image data includes data indicative of a known structure on which the vehicle is traveling;

determining, based on a first portion of the image data, information indicative of an appearance of the known structure in the environment;

determining, based on a second portion of the image data, information indicative of an appearance of an unknown object in the environment, wherein the second portion of the image data is different than the first portion of the image data;

comparing the information indicative of the appearance of the known structure in the environment with the information indicative of the appearance of the unknown object in the environment so as to determine whether the unknown object is part of the known structure; and

providing instructions to control the vehicle in the autonomous mode based on the comparison.

16. The non-transitory computer readable medium of claim 15, wherein the known structure comprises a road or a bridge.

17. The non-transitory computer readable medium of claim 15, wherein the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

determining a location of the known structure in the environment; and

causing the at least one sensor to obtain the image data based on the location.

18. The non-transitory computer readable medium of claim 15,

wherein the information indicative of the appearance of the known structure comprises one or more of a first color average, a first color-intensity, a first shadow presence, a first variation of color-intensity, or a first variation of color, and

wherein the information indicative of the appearance of the unknown structure comprises one or more of a second color average, a second color-intensity, a second shadow presence, a second variation of color-intensity, or a second variation of color.

19. The non-transitory computer readable medium of claim 18, wherein the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

comparing one or more of the first color average, the first color-intensity, the first shadow presence, the first variation of color-intensity, or the first variation of color with one or more of the second color average, the second color-intensity, the second shadow presence, the second variation of color-intensity, or the second variation of color.

20. The non-transitory computer readable medium of claim 19, wherein the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

obtaining second image data, wherein the second image data is different than the first image data;

determining, based on the second image data, information indicative of an appearance of a second unknown object in the environment, wherein the information indicative of the second unknown object comprises one or more of a third color average, a third color-intensity, a third shadow presence, a third variation of color-intensity, or a third variation of color; and

comparing the information indicative of the appearance of the second unknown object with the information indicative of the appearance of the known structure so as to determine whether the second unknown object is part of the known structure.