Use of Environmental Information to aid Image Processing for Autonomous Vehicles

• US 20140379247A1
• Filed: 06/24/2013
• Published: 12/25/2014
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Lo, Wan-Yen
• Granted Patent: US 9,145,139 B2
• Time in Patent Office: Days
• US Class Current: 701/301
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An autonomous vehicle may be configured to use environmental information for image processing. The vehicle may be configured to operate in an autonomous mode in an environment and may be operating substantially in a lane of travel of the environment. The vehicle may include a sensor configured to receive image data indicative of the environment. The vehicle may also include a computer system configured to compare environmental information indicative of the lane of travel to the image data so as to determine a portion of the image data that corresponds to the lane of travel of the environment. Based on the portion of the image data that corresponds to the lane of travel of the environment and by disregarding a remaining portion of the image data, the vehicle may determine whether an object is present in the lane, and based on the determination, provide instructions to control the vehicle in the autonomous mode in the environment.

Claims

1. A method comprising:

receiving, using a processor, image data indicative of an environment from at least one sensor of a vehicle, wherein the vehicle is configured to operate in an autonomous mode in the environment, and wherein the vehicle is substantially in a lane of travel of the environment;

comparing environmental information indicative of the lane of travel to the image data so as to determine a portion of the image data that corresponds to the lane of travel of the environment;

determining whether an object is present in the lane of travel based on the portion of the image data that corresponds to the lane of travel of the environment and by disregarding a remaining portion of the image data, wherein the remaining portion of the image data comprises image data other than the portion of the image data that corresponds to the lane of travel; and

based on the determination, providing instructions to control the vehicle in the autonomous mode in the environment.

2. The method of claim 1,

wherein the environmental information is further indicative of a second lane of travel of the environment, and the method further comprises:

comparing the environmental information to the image data;

based on the comparison, determining a second portion of the image data that corresponds to the second lane of travel of the environment, wherein the second portion of the image data is different than the portion of the image data, and wherein the second lane of travel of the environment is different than the lane of travel of the environment;

determining whether an object is present in the second lane of travel based on the second portion of the image data that corresponds to the second lane of travel of the environment and by disregarding a second remaining portion of the image data, wherein the second remaining portion of the image data comprises image data other than the second portion of the image data that corresponds to the second lane of travel; and

based on the determination, providing instructions to control the vehicle in the autonomous mode in the environment.

3. The method of claim 2,

wherein the environmental information comprises at least one of a location of the second lane of travel of the environment or a boundary of the second lane of travel of the environment, and

wherein comparing environmental information indicative of the lane of travel of the environment to the image data comprises:

comparing at least one of the location of the second lane of travel of the environment to the image data or the boundary of the second lane of travel of the environment to the image data.

4. The method of claim 2, wherein the second lane of travel of the environment is adjacent to the lane of travel of the environment.

5. The method of claim 1,

wherein the environmental information comprises at least one of a location of the lane of travel of the environment or a boundary of the lane of travel of the environment, and

wherein comparing environmental information indicative of the lane of travel of the environment to the image data comprises:

comparing at least one of the location of the lane of travel of the environment to the image data or a boundary of the lane of travel of the environment to the image data.

6. The method of claim 1, wherein the environmental information comprises an image map of the road in the environment.

7. The method of claim 1, wherein the environmental information comprises a heading of the vehicle in the environment.

8. A vehicle comprising:

a sensor configured to receive image data indicative of an environment, wherein the vehicle is configured to operate in an autonomous mode in the environment, and wherein the vehicle is substantially in a lane of travel of the environment;

a computer system, wherein the computer system is configured to;

compare environmental information indicative of the lane of travel to the image data so as to determine a portion of the image data that corresponds to the lane of travel of the environment;

determine whether an object is present in the lane of travel based on the portion of the image data that corresponds to the lane of travel of the environment and by disregarding a remaining portion of the image data, wherein the remaining portion of the image data comprises image data other than the portion of the image data that corresponds to the lane of travel; and

based on the determination, provide instructions to control the vehicle in the autonomous mode in the environment.

9. The vehicle of claim 8,

wherein the environmental information is further indicative of a second lane of travel of the environment, and the computer system is further configured to:

compare the environmental information to the image data;

based on the comparison, determine a second portion of the image data that corresponds to the second lane of travel of the environment, wherein the second portion of the image data is different than the portion of the image data, and wherein the second lane of travel of the environment is different than the lane of travel of the environment;

determining whether an object is present in the second lane of travel based on the second portion of the image data that corresponds to the second lane of travel of the environment and by disregarding a second remaining portion of the image data, wherein the second remaining portion of the image data comprises image data other than the second portion of the image data that corresponds to the second lane of travel; and

based on the determination, provide instructions to control the vehicle in the autonomous mode in the environment.

10. The vehicle of claim 9,

wherein the environmental information comprises at least one of a location of the second lane of travel of the environment or a boundary of the second lane of travel of the environment, and

wherein the computer system is further configured to:

compare at least one of the location of the second lane of travel of the environment to the image data or the boundary of the second lane of travel of the environment to the image data.

11. The vehicle of claim 10, wherein the second lane of travel of the environment is adjacent to the lane of travel of the environment.

12. The vehicle of claim 8,

wherein the environmental information comprises at least one of a location of the lane of travel of the environment or a boundary of the lane of travel of the environment, and

wherein the computer system is further configured to:

compare at least one of the location of the lane of travel of the environment to the image data or a boundary of the lane of travel of the environment to the image data.

13. The vehicle of claim 8, wherein the environmental information comprises an image map of the road in the environment.

19. The non-transitory computer readable medium of claim 13, wherein the environmental information comprises an image map of the road in the environment.

20. The non-transitory computer readable medium of claim 13, wherein the environmental information comprises a heading of the vehicle in the environment.

14. A non-transitory computer readable medium having stored therein instructions that when executed by a computer system in a vehicle, cause the computer system to perform functions comprising:

receiving image data indicative of an environment from at least one sensor of the vehicle, wherein the vehicle is configured to operate in an autonomous mode in the environment, and wherein the vehicle is substantially in a lane of travel of the environment;

comparing environmental information indicative of the lane of travel to the image data so as to determine a portion of the image data that corresponds to the lane of travel of the environment;

determining whether an object is present in the lane of travel based on the portion of the image data that corresponds to the lane of travel of the environment and by disregarding a remaining portion of the image data, wherein the remaining portion of the image data comprises image data other than the portion of the image data that corresponds to the lane of travel; and

based on the determination, providing instructions to control the vehicle in the autonomous mode in the environment.

15. The non-transitory computer readable medium of claim 14,

wherein the environmental information is further indicative of a second lane of travel of the environment, and the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

comparing the environmental information to the image data;

based on the comparison, determining a second portion of the image data that corresponds to the second lane of travel of the environment, wherein the second portion of the image data is different than the portion of the image data, and wherein the second lane of travel of the environment is different than the lane of travel of the environment;

determining whether an object is present in the second lane of travel based on the second portion of the image data that corresponds to the second lane of travel of the environment and by disregarding a second remaining portion of the image data, wherein the second remaining portion of the image data comprises image data other than the second portion of the image data that corresponds to the second lane of travel; and

based on the determination, providing instructions to control the vehicle in the autonomous mode in the environment.

16. The non-transitory computer readable medium of claim 15,

wherein the environmental information comprises at least one of a location of the second lane of travel of the environment or a boundary of the second lane of travel of the environment, and

wherein the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

comparing at least one of the location of the second lane of travel of the environment to the image data or the boundary of the second lane of travel of the environment to the image data.

17. The non-transitory computer readable medium of claim 14, wherein the second lane of travel of the environment is adjacent to the lane of travel of the environment.

18. The non-transitory computer readable medium of claim 14,

wherein the environmental information comprises at least one of a location of the lane of travel of the environment or a boundary of the lane of travel of the environment, and

wherein the instructions are further executable by the computer system in the vehicle to cause the computer system to perform functions comprising:

comparing at least one of the location of the lane of travel of the environment to the image data or a boundary of the lane of travel of the environment to the image data.