Ad hoc sensor arrays

• US 8,175,297 B1
• Filed: 07/06/2011
• Issued: 05/08/2012
• Est. Priority Date: 07/06/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ho, Harvey, Wong, Adrian
• Primary Examiner(s): FAULK, DEVONA E
• Application Number:
  US13/177,333
• Time in Patent Office: 307 Days
• Field of Search: 381/122, 381/92, 381/111, 381/77, 381/82, 381/311, 381/56, 381/58, 381/59, 381/74, 455/41.2, 455/41.3, 455/66.1
• US Class Current: 381/122
• CPC Class Codes: H04R 1/26 Spatial arrangements of separate transducers responsive to two or more frequency ranges
• CPC Class Codes: H04R 2201/023 Transducers incorporated in garment, rucksacks or the like
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Systems and methods for estimating audio at a requested location are presented. In one embodiment, the method includes receiving from a client device a request for audio at a requested location. The method further includes determining a location of a plurality of audio sensors, where the plurality of audio sensors are coupled to head-mounted devices in which a location of each of the plurality of audio sensors varies. The method further includes, based on the requested location and the location of the plurality of audio sensors, determining an ad hoc array of audio sensors, receiving audio sensed from audio sensors in the ad hoc array, and processing the audio sensed from audio sensors in the ad hoc array to produce an output substantially estimating audio at the requested location.

Claims

1. A method, comprising:

receiving from a client device a request for audio at a requested location;

determining a location of a plurality of audio sensors, wherein the plurality of audio sensors are coupled to head-mounted devices in which a location of each of the plurality of audio sensors varies;

based on the requested location and the location of the plurality of audio sensors, determining an ad hoc array of audio sensors, wherein determining the ad hoc array comprises;

selecting from a plurality of predefined environments a predefined environment in which the requested location is located;

identifying audio sensors in the plurality of audio sensors that are currently associated with the selected predefined environment;

determining a separation distance of the audio sensors currently associated with the selected predefined environment, wherein the separation distance for an audio sensor comprises a distance between the location of the audio sensor and the requested location; and

selecting for the ad hoc array audio sensors having a separation distance below a predetermined threshold;

receiving audio sensed from audio sensors in the ad hoc array; and

processing the audio sensed from audio sensors in the ad hoc array to produce an output substantially estimating audio at the requested location.

2. The method of claim 1, wherein receiving the request comprises receiving a set of coordinates identifying the requested location.

3. The method of claim 1, wherein determining the location of an audio sensor comprises at least one of querying the audio sensor for the location and receiving the location from the audio sensor.

4. The method of claim 1, wherein the location of an audio sensor comprises a location of the audio sensor relative to a known location.

5. The method of claim 1, wherein processing the audio sensed from audio sensors in the ad hoc array comprises processing the audio based on the location of each audio sensor in the ad hoc array.

6. The method of claim 5, wherein processing the audio based on the location of each audio sensor in the ad hoc array comprises:

for each audio sensor in the ad hoc array, delaying audio sensed by the audio sensor based on the separation distance of the audio sensor to produce a delayed audio signal; and

combining the delayed audio signals from each of the audio sensors in the ad hoc array.

7. The method of claim 1, wherein processing the audio sensed from audio sensors in the ad hoc array comprises using a beamforming process.

8. The method of claim 1, further comprising:

determining for audio sensors in the ad hoc array whether sensed audio may be received based on permissions set for the audio sensor.

9. The method of claim 1, further comprising:

receiving audio sensed by each audio sensor of the plurality of audio sensors; and

storing in memory the sensed audio, a corresponding location of the audio sensor where the audio was sensed, and a corresponding time at which the audio was sensed.

10. The method of claim 9, wherein the request further includes a time at which the audio at the requested location was sensed.

11. The method of claim 1, further comprising periodically determining an updated location of each audio sensor in the ad hoc array.

12. A server, comprising:

a first input interface configured to receive from a client device a request for audio at a requested location;

a second input interface configured to receive audio from audio sensors;

at least one processor; and

data storage comprising selection logic and processing logic, wherein the selection logic is executable by the at least one processor to;

determine a location of a plurality of audio sensors, wherein the plurality of audio sensors are coupled to head-mounted devices in which a location of each of the plurality of audio sensors varies;

based on the requested location and the location of the plurality of audio sensors, determine an ad hoc array of audio sensors, wherein determining the ad hoc array comprises;

selecting from a plurality of predefined environments a predefined environment in which the requested location is located;

identifying audio sensors in the plurality of audio sensors that are currently associated with the selected predefined environment;

determining a separation distance of the audio sensors currently associated with the selected predefined environment, wherein the separation distance for an audio sensor comprises a distance between the location of the audio sensor and the requested location; and

selecting for the ad hoc array audio sensors having a separation distance below a predetermined threshold,

wherein the processing logic is executable by the at least one processor to process the audio sensed from audio sensors in the ad hoc array to produce an output substantially estimating audio at the requested location.

13. The server of claim 12, wherein one or both of the first input interface and the second input interface is a wireless interface.

14. The server of claim 12, wherein the processing logic is further executable to process the audio based on the location of each audio sensor in the ad hoc array.

15. The server of claim 12, wherein the processing logic is further executable to request a given audio sensor in the ad hoc array to provide audio sensed from the audio sensor.

16. The server of claim 12, wherein the processing logic is further executable to:

receive audio sensed by each audio sensor of the plurality of audio sensors; and

store in the data storage the sensed audio, a corresponding location of the audio sensor where the audio was sensed, and a corresponding time at which the audio was sensed.

17. The server of claim 12, wherein the processing logic is further executable to periodically determine an updated location of each audio sensor in the ad hoc array.

18. The server of claim 12, wherein the server is configured to provide an instruction to control a direction of audio sensors in the ad hoc array.

19. The server of claim 12, further comprising an output interface configured to provide the output to the client device.

20. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform the functions of:

receiving from a client device a request for audio at a requested location;

determining a location of a plurality of audio sensors, wherein the plurality of audio sensors are coupled to head-mounted devices in which a location of each of the plurality of audio sensors varies;

based on the requested location and the location of the plurality of audio sensors, determining an ad hoc array of audio sensors, wherein determining the ad hoc array comprises;

selecting from a plurality of predefined environments a predefined environment in which the requested location is located;

identifying audio sensors in the plurality of audio sensors that are currently associated with the selected predefined environment;

determining a separation distance of the audio sensors currently associated with the selected predefined environment, wherein the separation distance for an audio sensor comprises a distance between the location of the audio sensor and the requested location; and

selecting for the ad hoc array audio sensors having a separation distance below a predetermined threshold;

receiving audio sensed from audio sensors in the ad hoc array; and

processing the audio sensed from audio sensors in the ad hoc array to produce an output substantially estimating audio at the requested location.