Controlling autonomous vehicle using audio data

• US 8,676,427 B1
• Filed: 10/11/2012
• Issued: 03/18/2014
• Est. Priority Date: 10/11/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Zhu, Jiajun
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): MAZZARA, ANNE MARIE
• Application Number:
  US13/649,646
• Time in Patent Office: 523 Days
• Field of Search: 701/23
• US Class Current: 701/23
• CPC Class Codes: B60W 10/06 including control of combustion engines
• CPC Class Codes: B60W 10/08 including control of electric propulsion units, e.g. motors or generators
• CPC Class Codes: B60W 10/10 including control of change-speed gearings
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/046 involving external transmission of data to or from the vehicle, e.g. via telemetry, satellite, Global Positioning System [GPS]
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2510/087 Temperature
• CPC Class Codes: B60W 2510/088 Inertia
• CPC Class Codes: B60W 2530/209 Fuel quantity remaining in tank
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/18 Propelling the vehicle
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G08G 1/0965 responding to signals from another vehicle, e.g. emergency vehicle

Abstract

A method includes receiving, at a computing system, audio data from one or more microphones associated with a vehicle. In one example, the audio data originates from a source external to the vehicle and the vehicle is configured to operate in an autonomous mode. The method also includes processing the audio data to generate directional data related to a direction of the source of the audio data relative to the vehicle and processing the audio data to generate identification data related to an identity of the source of the audio data. Further, the method includes controlling the vehicle in the autonomous mode in response to the directional data and the identification data, using the computing system.

Claims

1. A method comprising:

receiving, at a computing system, audio data from one or more microphones associated with a vehicle, wherein the audio data originates from a source external to the vehicle, and wherein the vehicle is configured to operate in an autonomous mode;

processing the audio data to generate directional data related to a direction of the source of the audio data relative to the vehicle;

processing the audio data to generate identification data related to an identity of the source of the audio data; and

controlling the vehicle in the autonomous mode in response to the directional data and the identification data, using the computing system.

2. The method of claim 1, wherein controlling the vehicle in the autonomous mode includes controlling movement of the vehicle.

3. The method of claim 1, wherein controlling the vehicle in the autonomous mode includes controlling one or more other sensors associated with the vehicle.

4. The method of claim 3, wherein controlling the one or more other sensors includes directing the one or more other sensors toward the source of the audio data to detect additional data relating to the source.

5. The method of claim 3, wherein the one or more sensors include at least one of an image sensor, a radar sensor, or a laser sensor.

6. The method of claim 1, further comprising processing the audio data to determine that the audio data originates from a source external to the vehicle.

7. The method of claim 1, wherein processing the audio data to generate identification data includes extracting audio features from the audio data and comparing the extracted audio features with stored audio features that are associated with known sources of audio data.

8. The method of claim 1, wherein the directional data relates to the source of the audio data approaching the vehicle and the identification data relates to an emergency vehicle, further wherein controlling the vehicle in response to the directional data and the identification data includes controlling the vehicle to reduce its speed and to pull over out of the way of the approaching emergency vehicle.

9. A non-transitory computer readable medium having stored therein instructions executable by a computer system to cause the computer system to perform functions, the functions comprising:

receiving audio data from one or more microphones associated with a vehicle, wherein the audio data originates from a source external to the vehicle, and wherein the vehicle is configured to operate in an autonomous mode;

processing the audio data to generate directional data related to a direction of the source of the audio data relative to the vehicle;

processing the audio data to generate identification data related to an identity of the source of the audio data; and

generating control data for controlling the vehicle in the autonomous mode in response to the directional data and the identification data.

10. The non-transitory computer readable medium of claim 9, wherein controlling the vehicle in the autonomous mode includes controlling movement of the vehicle.

11. The non-transitory computer readable medium of claim 9, wherein controlling the vehicle in the autonomous mode includes controlling one or more other sensors associated with the vehicle.

12. The non-transitory computer readable medium of claim 11, wherein controlling the one or more other sensors includes directing the one or more other sensors toward the source of the audio data to detect additional data relating to the source.

13. The non-transitory computer readable medium of claim 9, wherein the functions further comprise processing the audio data to determine that the audio data originates from a source external to the vehicle.

14. The non-transitory computer readable medium of claim 9, wherein processing the audio data to generate identification data includes extracting audio features from the audio data and comparing the extracted audio features with stored audio features that are associated with known sources of audio data.

15. The non-transitory computer readable medium of claim 9, wherein when the directional data relates to the source of the audio data approaching the vehicle and when the identification data relates to an emergency vehicle, controlling the vehicle in response to the directional data and the identification data includes controlling the vehicle to reduce its speed and to pull over out of the way of the approaching emergency vehicle.

16. A vehicle comprising:

one or more microphones for detecting audio data from a source external to the vehicle; and

a computing system configured to;

process the audio data to generate directional data related to a direction of the source of the audio data relative to the vehicle;

process the audio data to generate identification data related to an identity of the source of the audio data; and

control the vehicle in an autonomous mode in response to the directional data and the identification data.

17. The vehicle of claim 16, wherein the computing system is configured to control movement of the vehicle in the autonomous mode.

18. The vehicle claim 16, wherein the computing system is configured to control one or more other sensors associated with the vehicle in the autonomous mode.

19. The vehicle of claim 18, wherein the computing system is configured to control the one or more other sensors associated with the vehicle in the autonomous mode by directing the one or more other sensors toward the source of the audio data to detect additional data relating to the source.

20. The vehicle of claim 19, wherein the one or more sensors include at least one of an image sensor, a radar sensor, or a laser sensor.

21. The vehicle of claim 16, wherein the computing system is configured to process the audio data to determine that the audio data originates from a source external to the vehicle.

22. The vehicle of claim 16, wherein the computing system is configured to generate the identification data by extracting audio features from the audio data and comparing the extracted audio features with stored audio features that are associated with known sources of audio data.

23. The vehicle of claim 16, wherein when the directional data relates to the source of the audio data approaching the vehicle and when the identification data relates to an emergency vehicle, the processor is configured to control the vehicle to reduce its speed and to pull over out of the way of the approaching emergency vehicle.