Multi-listener stereo image array

• US 10,264,383 B1
• Filed: 09/23/2016
• Issued: 04/16/2019
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Johnson, Martin E., Crosby, Justin D., Saux, Tom-Davy W.
• Primary Examiner(s): Sniezek, Andrew L
• Application Number:
  US15/275,206
• Time in Patent Office: 935 Days
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Some embodiments provide a multi-listener stereo image array which provides multiple separate stereo images of audio content to each of multiple listeners while mitigating interference of audio signals which provide separate stereo images to separate listeners. The array can include sensors which can monitor the environment and can identify relative positions of various listeners and can further control the audio signal patterns generated by the drivers of the array to propagate towards positions associated with particular listeners and be at least partially restricted from propagating towards positions associated with other listeners.

Claims

1. An apparatus, comprising:

a speaker array which is configured to provide a plurality of listener-specific sound stages of audio content to each of a plurality of listeners, wherein the speaker array comprises;

a set of one or more sensor devices which are configured to generate one or more sensor data representations of an environment in which the speaker array is located;

a processor which is configured to identify a different position associated with each respective listener, of the plurality of listeners, based on processing the one or more sensor data representations; and

a set of drivers which are configured to at least partially collectively generate, for each respective listener of the plurality of listeners, a listener-specific audio signal pattern, comprising at least some of the audio content, which is shaped, based on a directivity index associated with the audio signal pattern, to at least partially inhibit an intensity of the audio signal pattern in a direction towards the position associated with another listener of the plurality of listeners,

wherein the set of drivers is configured to generate, at least partially concurrently, the listener-specific audio signal patterns in different directions towards the positions associated with each respective listener.

2. The apparatus of claim 1, wherein:

the speaker array is configured to, based on the identified positions of the plurality of listeners, adjustably control the shape of each listener-specific audio signal pattern, based on adjustably controlling the directivity index associated with each listener-specific audio signal pattern, such that each listener-specific audio signal pattern is shaped to;

provide at least a certain intensity of the audio signal pattern in a particular direction towards a particular identified position associated with a particular identified listener of the plurality of listeners, and

at least partially inhibit the intensity of the audio signal pattern in a particular direction towards a listener-specific position associated with a listener of the plurality of listeners.

3. The apparatus of claim 2, wherein, to adjustably control a shape of each listener-specific audio signal pattern based on the identified positions of the plurality of listeners, the speaker array is configured to, for each audio signal pattern:

adjustably control the directivity index associated with the listener-specific audio signal pattern so that the audio signal pattern comprises;

a maximized intensity of the audio signal pattern in the particular direction towards the particular identified position associated with the particular identified listener, and

a minimized intensity of the audio signal pattern in the particular direction towards the listener-specific position associated with the listener.

4. The apparatus of claim 3, wherein, to adjustably control a shape of each listener-specific audio signal pattern, the speaker array is configured to:

adjustably control audio signals generated by at least two drivers, of the set of drivers, such that the audio signals generated by the at least two drivers adjustably control a shape of at least one audio signal pattern to conform to a particular directivity index associated with the audio signal pattern, via beamforming.

5. The apparatus of claim 4, wherein:

the speaker array comprises at least one set of filter banks, wherein each filter bank comprises a plurality of filters which are each configured to provide a filtered output of at least one channel of the audio content to a separate driver of the set of drivers; and

to adjustably control audio signals generated by the at least two drivers, such that the audio signals generated by the at least two drivers adjustably control a shape of at least one audio signal pattern via beamforming, the speaker array is configured to;

determine a particular filter bank configuration which, when implemented, causes the at least one set of filter banks to provide filtered outputs which, when provided to the set of drivers, causes the at least two drivers to generate audio signals which collectively generate the at least one audio signal pattern which is associated with a particular directivity index via beamforming; and

adjustably control the at least one set of filter banks to cause the at least one set of filter banks to filter one or more channels of the audio content according to the particular filter bank configuration.

6. The apparatus of claim 5, wherein:

to determine the particular filter bank configuration, the speaker array is configured to select, of a plurality of sets of filter banks, a particular set of filter banks based on a determination that the particular set of filter banks is configured to provide a filtered output of at least one channel of the audio content to at least two drivers of the set of drivers to cause the at least two drivers to generate audio signals which collectively generate the at least one audio signal pattern via beamforming.

7. The apparatus of claim 2, wherein:

the set of drivers are positioned such that each separate driver, of the set of drivers, faces in a different direction; and

to, based on the identified positions of the plurality of listeners, adjustably control the shape of each listener-specific audio signal pattern, based on adjustably controlling the directivity index associated with each listener-specific audio signal pattern, the speaker array is configured to;

cause an individual driver, of the set of drivers, to generate the audio signal pattern, based on a determination that the individual driver is facing in a particular direction towards the particular identified position associated with the particular identified listener of the plurality of listeners and that the directivity index associated with the audio signal pattern at least meets a particular directivity threshold value.

8. A method, comprising:

configuring a speaker array, which comprises a set of drivers, to provide a plurality of listener-specific sound stages of audio content to each of a plurality of listeners, wherein the speaker array further comprises a set of one or more sensor devices which are configured to generate one or more sensor data representations of an environment in which the speaker array is located, and

wherein the configuring comprises;

identifying a different position associated with each respective listener, of the plurality of listeners, based on processing the one or more sensor data representations; and

adjustably controlling audio signals generated by at least two drivers of the set of drivers, such that the set of drivers at least partially collectively generate, for each respective listener of the plurality of listeners, a listener-specific audio signal pattern, comprising at least some of the audio content, which is shaped, based on a directivity index associated with the audio signal pattern, to at least partially inhibit an intensity of the audio signal pattern in a direction towards the position associated with another listener of the plurality of listeners,

wherein the at least two drivers are configured to generate, at least partially concurrently, the listener-specific audio signal patterns in different directions towards the positions associated with each respective listener.

9. The method of claim 8, wherein:

the configuring comprises;

adjustably controlling a shape of each listener-specific audio signal pattern, based on the identified positions of the plurality of listeners and adjustably controlling the directivity index associated with each listener-specific audio signal pattern, such that each listener-specific audio signal pattern is shaped to;

provide at least a certain intensity of the audio signal pattern in a particular direction towards a particular identified position associated with a particular identified listener of the plurality of listeners, and

at least partially inhibit the intensity of the audio signal pattern in a particular direction towards a listener-specific position associated with a listener of the plurality of listeners.

10. The method of claim 9, wherein adjustably controlling a shape of each listener-specific audio signal pattern based on the identified positions of the plurality of listeners and adjustably controlling the directivity index associated with each listener-specific audio signal pattern comprises, for each audio signal pattern:

adjustably controlling the directivity index associated with the listener-specific audio signal pattern so that the audio signal pattern comprises;

a maximized intensity of the audio signal pattern in the particular direction towards the particular identified position associated with the particular identified listener, and

a minimized intensity of the audio signal pattern in the particular direction towards the listener-specific position associated with the listener.

11. The method of claim 10, wherein adjustably controlling a shape of each listener-specific audio signal pattern comprises:

adjustably controlling audio signals generated by at least two drivers, of the set of drivers, such that the audio signals generated by the at least two drivers adjustably control a shape of at least one audio signal pattern to conform to a particular directivity index associated with the audio signal pattern, via beamforming.

12. The method of claim 11, wherein:

the speaker array comprises at least one set of filter banks, wherein each filter bank comprises a plurality of filters which are each configured to provide a filtered output of at least one channel of the audio content to a separate driver of the set of drivers; and

adjustably controlling audio signals generated by the at least two drivers, such that the audio signals generated by the at least two drivers adjustably control a shape of at least one audio signal pattern via beamforming, comprises;

determining a particular filter bank configuration which, when implemented, causes the at least one set of filter banks to provide filtered outputs which, when provided to the set of drivers, causes the at least two drivers to generate audio signals which collectively generate the at least one audio signal pattern via beamforming; and

adjustably controlling the at least one set of filter banks to cause the at least one set of filter banks to filter one or more channels of the audio content according to the particular filter bank configuration.

13. The method of claim 12, wherein:

determine the particular filter bank configuration comprises selecting, of a plurality of sets of filter banks comprised in the speaker array, a particular set of filter banks based on a determination that the particular set of filter banks is configured to provide a filtered output of at least one channel of the audio content to at least two drivers of the set of drivers to cause the at least two drivers to generate audio signals which collectively generate the at least one audio signal pattern via beamforming.

14. The method of claim 9, wherein:

the set of drivers are positioned such that each separate driver, of the set of drivers, faces in a different direction; and

adjustably controlling a shape of each listener-specific audio signal pattern, based on the identified positions of the plurality of listeners, comprises;

causing an individual driver, of the set of drivers, to generate the audio signal pattern, based on a determination that the individual driver is facing in a particular direction towards the particular identified position associated with the particular identified listener of the plurality of listeners and that the directivity index associated with the audio signal pattern at least meets a particular directivity threshold value.

15. A non-transitory computer readable medium storing a program of instructions which, when executed by at least one computer system, cause the at least one computer system to:

configure a speaker array, which comprises at least one set of filter banks and a set of drivers which generate audio signals based on filtered outputs generated by the at least one set of filter banks, to provide a plurality of listener-specific sound stages of audio content to each of a plurality of listeners, wherein the speaker array further comprises a set of one or more sensor devices which are configured to generate one or more sensor data representations of an environment in which the speaker array is located,

wherein the configuring comprises;

identifying a different position associated with each respective listener, of the plurality of listeners, based on processing the one or more sensor data representations; and

controllably adjusting a filtering of at least one audio channel of the audio content, by the at least one set of filter banks, to cause the at least one set of filter banks to provide filtered outputs of at least one channel of the audio content to the set of drivers, which causes at least two drivers of the set of drivers to generate audio signals which collectively and at least partially concurrently provide, for each respective listener of the plurality of listeners, a listener-specific audio signal pattern, comprising at least some of the audio content, which is shaped, based on a directivity index associated with the audio signal pattern, to at least partially inhibit an intensity of the audio signal pattern in a direction towards the position associated with another listener of the plurality of listeners.

16. The non-transitory computer readable medium of claim 15, wherein:

the configuring comprises;

controllably adjusting the filtering of at least one audio channel of the audio content, by the at least one set of filter banks, such that each listener-specific audio signal pattern is shaped, based on a directivity index associated with the audio signal pattern and based on the identified positions of the plurality of listeners, to;

provide at least a certain intensity of the audio signal pattern in a particular direction towards a particular identified position associated with a particular identified listener of the plurality of listeners, and

at least partially inhibit the intensity of the audio signal pattern in a particular direction towards a listener-specific position associated with a listener of the plurality of listeners.

17. The non-transitory computer readable medium of claim 16, wherein controllably adjusting a filtering of at least one audio channel of the audio content, by the at least one set of filter banks, based on the identified positions of the plurality of listeners comprises, for each audio signal pattern:

controllably adjusting a filtering of at least one audio channel of the audio content, by the at least one set of filter banks, so that the audio signal pattern comprises;

a maximized intensity of the audio signal pattern in the particular direction towards the particular identified position associated with the particular identified listener, and

a minimized intensity of the audio signal pattern in the particular direction towards the listener-specific position associated with the listener.

18. The non-transitory computer readable medium of claim 17, wherein, the program of instructions, when executed by at least one computer system, cause the at least one computer system to:

controllably adjust the filtering of at least one audio channel of the audio content, by the at least one set of filter banks, to cause the at least one set of filter banks to provide filtered outputs of at least one channel of the audio content to the set of drivers, which causes at least two drivers of the set of drivers to generate audio signals which collectively provide, for each respective listener of the plurality of listeners, a listener-specific audio signal pattern via beamforming.

19. The non-transitory computer readable medium of claim 18, wherein:

controllably adjusting the filtering of at least one audio channel of the audio content, by the at least one set of filter banks, comprises;

determining a particular filter bank configuration which, when implemented, causes the at least one set of filter banks to provide outputs which, when provided to the set of drivers, causes the at least two drivers to generate audio signals which collectively generate the at least one audio signal pattern via beamforming; and

adjustably controlling the at least one set of filter banks to cause the at least one set of filter banks to filter at least one channel of the audio content according to the particular filter bank configuration.

20. The non-transitory computer readable medium of claim 16, wherein:

controllably adjusting the filtering of at least one audio channel of the audio content, by the at least one set of filter banks, based on a directivity index associated with the audio signal pattern and based on the identified positions of the plurality of listeners, comprises;

causing an individual driver, of the set of drivers, to generate the audio signal pattern, based on a determination that the individual driver is facing in a particular direction towards the particular identified position associated with the particular identified listener of the plurality of listeners and that the directivity index associated with the audio signal pattern at least meets a particular directivity threshold value.