VEHICULAR THREAT DETECTION BASED ON AUDIO SIGNALS

• US 20130142347A1
• Filed: 01/31/2012
• Published: 06/06/2013
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Lowell L. Wood Jr, William H. Gates
• Inventors: Lord, Richard T., Lord, Robert W., Myhrvold, Nathan P., Tegreene, Clarence T., Hyde, Roderick A., Wood, Lowell L. Jr., Ishikawa, Muriel Y., Wood, Victoria Y.H., Whitmer, Charles, Bahl, Paramvir, Burger, Douglas C., Chandra, Ranveer, Gates, William H. III, Holman, Paul, Kare, Jordin T., Mundie, Craig J., Paek, Tim, Tan, Desney S., Zhong, Lin, Dyor, Matthew G.
• Granted Patent: US 9,107,012 B2
• Time in Patent Office: Days
• US Class Current: 381/58
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2460/07 Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 29/005 Microphone arrays

Abstract

Techniques for ability enhancement are described. Some embodiments provide an ability enhancement facilitator system (“AEFS”) configured to enhance a user'"'"'s ability to operate or function in a transportation-related context as a pedestrian or a vehicle operator. In one embodiment, the AEFS is configured perform vehicular threat detection based at least in part on analyzing audio signals. An example AEFS receives data that represents an audio signal emitted by a vehicle. The AEFS analyzes the audio signal to determine vehicular threat information, such as that the vehicle may collide with the user. The AEFS then informs the user of the determined vehicular threat information, such as by transmitting a warning to a wearable device configured to present the warning to the user.

Claims

1. A method for enhancing ability in a transportation-related context, the method comprising:

receiving data representing an audio signal obtained in proximity to a user, the audio signal emitted by a first vehicle;

determining vehicular threat information based at least in part on the data representing the audio signal; and

presenting the vehicular threat information via a wearable device of the user.

2. The method of claim 1, wherein the receiving data representing an audio signal includes:

receiving data obtained at a microphone array that includes multiple microphones.

3. The method of claim 2, wherein the receiving data obtained at a microphone array includes:

receiving data obtained at a microphone array, the microphone array coupled to a vehicle of the user.

4. The method of claim 2, wherein the receiving data obtained at a microphone array includes:

receiving data obtained at a microphone array, the microphone array coupled to the wearable device.

5. The method of claim 1, wherein the determining vehicular threat information includes:

determining a position of the first vehicle.

6. The method of claim 1, wherein the determining vehicular threat information includes:

determining a velocity of the first vehicle.

7. The method of claim 1, wherein the determining vehicular threat information includes:

determining a direction of travel of the first vehicle.

8. The method of claim 1, wherein the determining vehicular threat information includes:

determining whether the first vehicle is approaching the user.

9. The method of claim 1, wherein the determining vehicular threat information includes:

performing acoustic source localization to determine a position of the first vehicle based on multiple audio signals received via multiple microphones.

10. The method of claim 9, wherein the performing acoustic source localization includes:

receiving an audio signal via a first one of the multiple microphones, the audio signal representing a sound created by the first vehicle;

receiving the audio signal via a second one of the multiple microphones; and

determining the position of the first vehicle by determining a difference between an arrival time of the audio signal at the first microphone and an arrival time of the audio signal at the second microphone.

11. The method of claim 9, wherein the performing acoustic source localization includes:

triangulating the position of the first vehicle based on a first and second angle, the first angle measured between a first one of the multiple microphones and the first vehicle, the second angle measured between a second one of the multiple microphones and the first vehicle.

12. The method of claim 1, wherein the determining vehicular threat information includes:

performing a Doppler analysis of the data representing the audio signal to determine whether the first vehicle is approaching the user.

13. The method of claim 12, wherein the performing a Doppler analysis includes:

determining whether frequency of the audio signal is increasing or decreasing.

14. The method of claim 1, wherein the determining vehicular threat information includes:

performing a volume analysis of the data representing the audio signal to determine whether the first vehicle is approaching the user.

15. The method of claim 14, wherein the performing a volume analysis includes:

determining whether volume of the audio signal is increasing or decreasing.

16. The method of claim 1, wherein the determining vehicular threat information includes:

determining the vehicular threat information based on gaze information associated with the user.

17. The method of claim 16, further comprising:

receiving an indication of a direction in which the user is looking;

determining that the user is not looking towards the first vehicle; and

in response to determining that the user is not looking towards the first vehicle, directing the user to look towards the first vehicle.

18. The method of claim 1, further comprising:

identifying multiple threats to the user;

identifying a first one of the multiple threats that is more significant than at least one other of the multiple threats; and

causing the user to avoid the first one of the multiple threats.

19. The method of claim 1, further comprising:

determining vehicular threat information related to factors other than ones related to the first vehicle.

20. The method of claim 19, wherein the determining vehicular threat information related to factors other than ones related to the first vehicle includes at least one of:

determining that poor driving conditions exist;

determining that there is stalled or slow traffic in proximity to the user, determining that poor surface conditions exist on a roadway traveled by the user, and/or determining that there is an animal in proximity to the user.

21. The method of claim 19, wherein the determining vehicular threat information related to factors other than ones related to the first vehicle includes:

determining that a limited visibility condition exists.

25. The method of claim 19, wherein the determining vehicular threat information related to factors other than ones related to the first vehicle includes:

determining that there is an accident in proximity to the user.

27. The method of claim 1, wherein the determining vehicular threat information includes:

determining the vehicular threat information based on kinematic information.

28. The method of claim 27, wherein the determining the vehicular threat information based on kinematic information includes:

determining the vehicular threat information based on information about position, velocity, and/or acceleration of the user obtained from sensors in the wearable device.

29. The method of claim 27, wherein the determining the vehicular threat information based on kinematic information includes:

determining the vehicular threat information based on information about position, velocity, and/or acceleration of the user obtained from devices in a vehicle of the user.

30. The method of claim 27, wherein the determining the vehicular threat information based on kinematic information includes:

determining the vehicular threat information based on information about position, velocity, and/or acceleration of the first vehicle.

31. The method of claim 1, wherein the presenting the vehicular threat information includes:

presenting the vehicular threat information via an audio output device of the wearable device.

32. The method of claim 1, wherein the presenting the vehicular threat information includes:

presenting the vehicular threat information via a visual display device of the wearable device.

33. The method of claim 32, wherein the presenting the vehicular threat information via a visual display device includes:

displaying an indicator that instructs the user to look towards the first vehicle.

35. The method of claim 1, wherein the presenting the vehicular threat information includes at least one of:

directing the user to accelerate, directing the user to decelerate, and/or directing the user to turn.

38. The method of claim 1, further comprising:

transmitting to the first vehicle a warning based on the vehicular threat information.

39. The method of claim 1, further comprising:

presenting the vehicular threat information via an output device of a vehicle of the user, the output device including a visual display and/or an audio speaker.

40. The method of claim 1, wherein the wearable device is on of a helmet, goggles, eyeglasses, or a hat worn by the user.

47. The method of claim 1, wherein the presenting the vehicular threat information includes:

presenting the vehicular threat information via audio speakers that are part of at least one of earphones, a headset, earbuds, and/or a hearing aid.

48. The method of claim 1, further comprising:

performing the receiving data representing an audio signal, the determining vehicular threat information, and/or the presenting the vehicular threat information on a computing device in the wearable device of the user.

49. The method of claim 1, further comprising:

performing the receiving data representing an audio signal, the determining vehicular threat information, and/or the presenting the vehicular threat information on a road-side computing system; and

transmitting the vehicular threat information from the road-side computing system to the wearable device of the user.

50. The method of claim 1, further comprising:

performing the receiving data representing an audio signal, the determining vehicular threat information, and/or the presenting the vehicular threat information on a computing system in the first vehicle; and

transmitting the vehicular threat information from the computing system to the wearable device of the user.

51. The method of claim 1, further comprising:

performing the receiving data representing an audio signal, the determining vehicular threat information, and/or the presenting the vehicular threat information on a computing system in a second vehicle, wherein the user is not traveling in the second vehicle; and

transmitting the vehicular threat information from the computing system to the wearable device of the user.

52. The method of claim 1, further comprising:

receiving data representing a visual signal that represents the first vehicle, the receiving including receiving an image of the first vehicle obtained by a camera of a vehicle operated by the user and/or receiving an image of the first vehicle obtained by a camera of the wearable device; and

determining the vehicular threat information based further on the data representing the visual signal, the determining including identifying the first vehicle in an image represented by the data representing a visual signal and/or determining whether the first vehicle is moving towards the user based on multiple images represented by the data representing the visual signal.

57. The method of claim 1, further comprising:

receiving data representing the first vehicle obtained at a road-based device; and

determining the vehicular threat information based further on the data representing the first vehicle.

58. The method of claim 57, wherein the receiving data representing the first vehicle obtained at a road-based device includes at least one of:

receiving the data from a sensor deployed at an intersection;

receiving an image of the first vehicle from a camera deployed at an intersection;

receiving ranging data from a range sensor deployed at an intersection, the ranging data representing a distance between the first vehicle and the intersection; and

/or receiving data from an induction loop deployed in a road surface, the induction loop configured to detect the presence and/or velocity of the first vehicle.

62. The method of claim 57, wherein the determining the vehicular threat information based further on the data representing the first vehicle includes:

identifying the first vehicle in an image obtained from the road-based sensor.

63. The method of claim 57, wherein the determining the vehicular threat information based further on the data representing the first vehicle includes:

determining a trajectory of the first vehicle based on multiple images obtained from the road-based device.

64. The method of claim 1, further comprising:

receiving data representing vehicular threat information relevant to a second vehicle, the second vehicle not being used for travel by user; and

determining the vehicular threat information based on the data representing vehicular threat information relevant to the second vehicle.

65. The method of claim 64, wherein the receiving data representing vehicular threat information relevant to a second vehicle includes:

receiving from the second vehicle at least one of;

an indication of stalled or slow traffic encountered by the second vehicle, an indication of poor driving conditions experienced by the second vehicle, an indication that the first vehicle is driving erratically, and/or an image of the first vehicle.

69. The method of claim 1, further comprising:

transmitting the vehicular threat information to a second vehicle.

70. The method of claim 69, wherein the transmitting the vehicular threat information to a second vehicle includes:

transmitting the vehicular threat information to an intermediary server system for distribution to other vehicles in proximity to the user.

22-24. -24. (canceled)

26. (canceled)

34. (canceled)

36. (canceled)

37. (canceled)

41-46. -46. (canceled)

53-56. -56. (canceled)

59-61. -61. (canceled)

66-68. -68. (canceled)

71-210. -210. (canceled)