OPTIMIZING THE PERFORMANCE OF AN AUDIO PLAYBACK SYSTEM WITH A LINKED AUDIO/VIDEO FEED

• US 20170272886A1
• Filed: 08/18/2014
• Published: 09/21/2017
• Est. Priority Date: 08/18/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Family, Afrooz, Johnson, Martin E., Paquier, Baptiste P.
• Granted Patent: US 10,542,365 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04R 2205/022 Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
• CPC Class Codes: H04R 2205/024 Positioning of loudspeaker enclosures for spatial sound reproduction
• CPC Class Codes: H04R 2227/007 Electronic adaptation of audio signals to reverberation of the listening space for PA
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence

Abstract

An audio system is provided that efficiently detects speaker arrays and configures the speaker arrays to output sound. In this system, a computing device may record the addresses and/or types of speaker arrays on a shared network while a camera captures video of a listening area, including the speaker arrays. The captured video may be analyzed to determine the location of the speaker arrays, one or more users, and/or the audio source in the listening area. While capturing the video, the speaker arrays may be driven to sequentially emit a series of test sounds into the listening area and a user may be prompted to select which speaker arrays in the captured video emitted each of the test sounds. Based on these inputs from the user, the computing device may determine an association between the speaker arrays on the shared network and the speaker arrays in the captured video.

Claims

1. A method for detecting and configuring speaker arrays, comprising:

detecting, by a computing device, a first set of one or more speaker arrays on a shared network with the computing device;

capturing, by the computing device, video of a listening area in which a second set of one or more speaker arrays are located;

determining a location of each speaker array in the second set of one or more speaker arrays in the listening area based on the captured video;

driving each speaker array in the first set of speaker arrays sequentially to emit one or more test sounds; and

determining a location of each speaker array in the first set of one or more speaker arrays in the listening area based on the emitted test sounds and the determined locations of the second set of one or more speaker arrays.

2. The method of claim 1, wherein determining the location of each speaker array in the first set of one or more speaker arrays comprises:

associating each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays based on the test sounds and the captured video.

3. The method of claim 2, wherein associating each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays comprises:

prompting a user to select which speaker array in the second set of one or more speaker arrays in the captured video emitted each test sound, wherein the selected speaker arrays in the second set of one or more speaker arrays are associated with corresponding speaker arrays in the first set of one or more speaker arrays that were driven to emit test sounds.

4. The method of claim 2, wherein the location of each speaker array in the second set of one or more speaker arrays in the listening area includes one or more of 1) a distance between each speaker array and another speaker array in the captured video, 2) a distance between each speaker array and the user, and 3) a distance between each speaker array and an audio source.

5. The method of claim 1, further comprising:

recording each of the test sounds emitted by the first set of one or more speaker arrays; and

determining characteristics of the listening area based on the recorded tests sounds.

6. The method of claim 5, wherein the characteristics of the listening area include the reverberation characteristics of the listening area.

7. The method of claim 1, further comprising:

assigning roles to each speaker array in the first set of one or more speaker arrays based on

     1) the association of each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays and

     2) the determined location of each speaker array in the second set of one or more speaker arrays.

8. The method of claim 7, wherein the roles include channel assignment for an audio piece played by an audio source.

9. A computing device for detecting and configuring speaker arrays, comprising:

a hardware processor; and

a memory unit for storing instructions, which when executed by the hardware processor;

detect a first set of one or more speaker arrays on a shared network with the computer;

capture video of a listening area in which a second set of one or more speaker arrays are located;

determine a location of each speaker array in the second set of one or more speaker arrays in the listening area based on the captured video;

drive each speaker array in the first set of speaker arrays to sequentially emit one or more test sounds; and

determine a location of each speaker array in the first set of one or more speaker arrays in the listening area based on the emitted test sounds and the determined locations of the second set of one or more speaker arrays.

10. The computing device of claim 9, wherein determining the location of each speaker array in the first set of one or more speaker arrays comprises:

associating each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays based on the test sounds and the captured video.

11. The device of claim 10, wherein associating each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays comprises:

prompting a user to select which speaker array in the second set of one or more speaker arrays in the captured video emitted each test sound, wherein the selected speaker arrays in the second set of one or more speaker arrays are associated with corresponding speaker arrays in the first set of one or more speaker arrays that were driven to emit test sounds.

12. The device of claim 10, wherein the location of each speaker array in the second set of one or more speaker arrays in the listening area includes one or more of 1) a distance between each speaker array and another speaker array in the captured video, 2) a distance between each speaker array and the user, and 3) a distance between each speaker array and an audio source.

13. The device of claim 9, further comprising:

recording each of the test sounds emitted by the first set of one or more speaker arrays; and

determining characteristics of the listening area based on the recorded tests sounds.

14. The device of claim 13, wherein the characteristics of the listening area include the reverberation characteristics of the listening area.

15. The device of claim 9, further comprising:

assigning roles to each speaker array in the first set of one or more speaker arrays based on

     1) the association of each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays and

     2) the determined location of each speaker array in the second set of one or more speaker arrays.

16. The device of claim 15, wherein the roles include channel assignment for an audio piece played by an audio source.

17. An article of manufacture for detecting and configuring speaker arrays, comprising:

a non-transitory machine-readable storage medium that stores instructions which, when executed by a processor in a computer,

detect a first set of one or more speaker arrays on a shared network with the computer;

capture video of a listening area in which a second set of one or more speaker arrays are located;

determine a location of each speaker array in the second set of one or more speaker arrays in the listening area based on the captured video;

generate drive signals for each speaker array in the first set of speaker arrays to sequentially emit one or more test sounds; and

determine a location of each speaker array in the first set of one or more speaker arrays in the listening area based on the emitted test sounds and the determined locations of the second set of one or more speaker arrays.

18. The article of manufacture of claim of claim 17, wherein the non-transitory machine-readable storage medium stores further instructions which, when executed by the processor:

associate each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays based on the test sounds and the captured video.

19. The article of manufacture of claim of claim 18, wherein the non-transitory machine-readable storage medium stores further instructions which, when executed by the processor:

prompt a user to select which speaker array in the second set of one or more speaker arrays in the captured video emitted each test sound, wherein the selected speaker arrays in the second set of one or more speaker arrays are associated with corresponding speaker arrays in the first set of one or more speaker arrays that were driven to emit test sounds.

20. The article of manufacture of claim of claim 18, wherein the location of each speaker array in the second set of one or more speaker arrays in the listening area includes one or more of 1) a distance between each speaker array and another speaker array in the captured video, 2) a distance between each speaker array and the user, and 3) a distance between each speaker array and an audio source.

21. The article of manufacture of claim of claim 17, wherein the non-transitory machine-readable storage medium stores further instructions which, when executed by the processor:

record each of the test sounds emitted by the first set of one or more speaker arrays; and

determine characteristics of the listening area based on the recorded tests sounds.

22. The article of manufacture of claim of claim 21, wherein the characteristics of the listening area include the reverberation characteristics of the listening area.

23. The article of manufacture of claim of claim 17, wherein the non-transitory machine-readable storage medium stores further instructions which, when executed by the processor:

assign roles to each speaker array in the first set of one or more speaker arrays based on

     1) the association of each speaker array in the first set of one or more speaker arrays with a speaker array in the second set of one or more speaker arrays and

     2) the determined location of each speaker array in the second set of one or more speaker arrays.

24. The article of manufacture of claim of claim 23, wherein the roles include channel assignment for an audio piece played by an audio source.