Collaborative sound system

• US 9,154,877 B2
• Filed: 03/14/2013
• Issued: 10/06/2015
• Est. Priority Date: 11/28/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kim, Lae-Hoon, Xiang, Pei
• Primary Examiner(s): BERNARDI, BRENDA C
• Application Number:
  US13/831,515
• Publication Number:
  US 20140146970A1
• Time in Patent Office: 936 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04R 2205/024 Positioning of loudspeaker enclosures for spatial sound reproduction
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 5/00 Stereophonic arrangements stereophonic pick-ups H04R9/16, H04R11/12, H04R17/08, H04R19/10
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection

Abstract

In general, techniques are described for forming a collaborative sound system. A headend device comprising one or more processors may perform the techniques. The processors may be configured to identify mobile devices that each includes a speaker and that are available to participate in a collaborative surround sound system. The processors may configure the collaborative surround sound system to utilize the speaker of each of the mobile devices as one or more virtual speakers of this system and then render audio signals from an audio source such that when the audio signals are played by the speakers of the mobile devices the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system. The processors may then transmit the processed audio signals rendered to the mobile device participating in the collaborative surround sound system.

Claims

1. A method comprising:

identifying one or more mobile devices, each of the one or more mobile devices is separate from a headend device of a collaborative surround sound system, includes a speaker, and is available to participate in the collaborative surround sound system;

configuring the collaborative surround sound system to utilize the speaker of each of the one or more mobile devices as one or more virtual speakers of the collaborative surround sound system;

receiving mobile device data from one of the identified one or more mobile devices that specifies a power level of the corresponding one of the identified mobile devices;

rendering audio signals from an audio source based on the determined power level of the identified mobile device so as to control playback of the audio signals from the audio source to accommodate the power level of the mobile device and such that when the audio signals are played by the speakers of the one or more mobile devices, the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system; and

transmitting the processed audio signals rendered from the audio source to each of the one or more mobile devices participating in the collaborative surround sound system.

2. The method of claim 1, wherein the one or more virtual speakers of the collaborative surround sound system appear to be placed in a location different than a location of at least one of the one or more mobile devices.

3. The method of claim 1, wherein configuring the collaborative surround sound system comprises identifying speaker sectors at which each of the virtual speakers of the collaborative surround sound system are to appear to originate the audio playback of the audio signals, and

wherein rendering the audio signals comprises rendering the audio signals from the audio source such that, when the audio signals are played by the speakers of the one or more mobile devices, the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system placed in a location within the corresponding identified speaker sectors.

4. The method of claim 1, further comprising receiving mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein configuring the collaborative surround sound system comprises configuring the collaborative surround sound system based on the associated mobile device data to utilize the speaker of each of the one or more mobile devices as the one or more virtual speakers of the collaborative surround sound system.

5. The method of claim 1, further comprising receiving mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein configuring the collaborative surround sound system comprises:

determining that the one of the identified mobile devices is not in a specified location for playing the audio signals rendered from the audio source based on the location of the one of the identified mobile devices determined based on the mobile device data; and

prompting a user of the one of the identified mobile devices to re-position the one of the identified mobile devices to modify playback of the audio by the one of the identified mobile devices.

6. The method of claim 1, further comprising receiving mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices, wherein rendering the audio signals comprises:

configuring an audio pre-processing function based on the location of one of the identified mobile devices so as to avoid prompting a user to move the one of the identified mobile devices; and

performing the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the location of the one of the identified mobile devices, and

wherein transmitting the audio signals comprises transmitting at least the pre-processed portion of the audio signals rendered from the audio source to the one of the identified mobile devices.

7. The method of claim 1, further comprising receiving mobile device data from one of the identified one or more mobile devices that specifies one or more speaker characteristics of the speaker included within one of the identified mobile devices,

wherein rendering the audio signals comprises:

configuring an audio pre-processing function by which to process the audio signals from the audio source based on the one or more speaker characteristics; and

performing the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the one or more speaker characteristics of the speaker included within the one of the identified mobile devices, and

wherein transmitting the audio signals comprises transmitting at least the pre-processed portion of the audio signals to the one of the identified mobile devices.

8. The method of claim 1, further comprising receiving mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the mobile device data specifies one or more of a location of the corresponding one of the identified mobile devices, a frequency response of the speaker included within the corresponding one of the identified mobile devices, a maximum allowable sound reproduction level of the speaker included within the corresponding one of the identified mobile devices, the power level of the corresponding one of the identified mobile devices, a sync status of the corresponding one of the identified mobile devices, and a headphone status the corresponding one of the identified mobile devices.

9. The method of claim 1, further comprising determining that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals rendered from the audio source, wherein rendering the audio signals from the audio source comprises rendering the audio signals to reduce an amount of power required by the corresponding one of the mobile devices to play the audio signals based on the determination that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals.

10. The method of claim 1, further comprising receiving mobile device data from one of the identified one or more mobile devices that specifies the power level of the corresponding one of the identified mobile devices, and

wherein rendering the audio signals from the audio source comprises one or more of:

adjusting a volume of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device;

cross-mixing the audio signals to be played by the corresponding one of the mobile devices with the audio signals to be played by one or more of the remaining mobile devices to accommodate the power level of the mobile device; and

reducing at least some range of frequencies of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device.

11. The method of claim 1, wherein the audio source comprises one of a higher order ambisonic audio source data, a multi-channel audio source data and an object-based audio source data.

12. A headend device comprising:

one or more processors configured to;

identify one or more mobile devices, each of the one or more mobile devices is separate from the headend device, includes a speaker, and is available to participate in a collaborative surround sound system;

configure the collaborative surround sound system to utilize the speaker of each of the one or more mobile devices as one or more virtual speakers of the collaborative surround sound system;

a receiver configured to receive mobile device data from one of the identified one or more mobile devices that specifies a power level of the corresponding one of the identified mobile devices,

wherein the one or more processors are further configured to render audio signals from an audio source based on the determined power level of the identified mobile device so as to control playback of the audio signals from the audio source to accommodate the power level of the mobile device and such that when the audio signals are played by the speakers of the one or more mobile devices the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system; and

a transmitter configured to transmit the processed audio signals rendered from the audio source to each of the one or more mobile devices participating in the collaborative surround sound system.

13. The headend device of claim 12, wherein the one or more virtual speakers of the collaborative surround sound system appear to be placed in a location different than a location of at least one of the one or more mobile devices.

14. The headend device of claim 12,

wherein the one or more processors are further configured to, when configuring the collaborative surround sound system, identify speaker sectors at which each of the virtual speakers of the collaborative surround sound system are to appear to originate the audio playback of the audio signals, and

wherein the one or more processors are further configured to, when rendering the audio signals, render the audio signals from the audio source such that, when the audio signals are played by the speakers of the one or more mobile devices, the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system placed in a location within the corresponding identified speaker sectors.

15. The headend device of claim 12,

wherein the one or more processors are further configured to receive mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the one or more processors are further configured to, when configuring the collaborative surround sound system, configure the collaborative surround sound system based on the associated mobile device data to utilize the speaker of each of the one or more mobile devices as the one or more virtual speakers of the collaborative surround sound system.

16. The headend device of claim 12,

wherein the one or more processors are further configured to receive mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein the one or more processors are further configured to, when configuring the collaborative surround sound system, determine that the one of the identified mobile devices is not in a specified location for playing the audio signals rendered from the audio source based on the location of the one of the identified mobile devices determined based on the mobile device data, and prompt a user of the one of the identified mobile devices to re-position the one of the identified mobile devices to modify playback of the audio by the one of the identified mobile devices.

17. The headend device of claim 12,

wherein the one or more processors are further configured to receive mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein the one or more processors are further configured to, when rendering the audio signals, configure an audio pre-processing function based on the location of one of the identified mobile devices so as to avoid prompting a user to move the one of the identified mobile devices, and perform the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the location of the one of the identified mobile devices, and

wherein the transmitter is further configured to, when transmitting the audio signals, transmit at least the pre-processed portion of the audio signals rendered from the audio source to the one of the identified mobile devices.

18. The headend device of claim 12,

wherein the receiver is further configured to receive mobile device data from one of the identified one or more mobile devices that specifies one or more speaker characteristics of the speaker included within one of the identified mobile devices,

wherein the one or more processors are further configured to, when rendering the audio signals, configure an audio pre-processing function by which to process the audio signals from the audio source based on the one or more speaker characteristics, and perform the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the one or more speaker characteristics of the speaker included within the one of the identified mobile devices, and

wherein the transmitter is further configured to, when transmitting the audio signals, transmit at least the pre-processed portion of the audio signals to the one of the identified mobile devices.

19. The headend device of claim 12,

wherein the receiver is further configured to receive mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the mobile device data specifies one or more of a location of the corresponding one of the identified mobile devices, a frequency response of the speaker included within the corresponding one of the identified mobile devices, a maximum allowable sound reproduction level of the speaker included within the corresponding one of the identified mobile devices, the power level of the corresponding one of the identified mobile devices, a sync status of the corresponding one of the identified mobile devices, and a headphone status the corresponding one of the identified mobile devices.

20. The headend device of claim 12, wherein the one or more processors are further configured to determine that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals rendered from the audio source, wherein rendering the audio signals from the audio source comprises rendering the audio signals to reduce an amount of power required by the corresponding one of the mobile devices to play the audio signals based on the determination that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals.

21. The headend device of claim 12,

wherein the receiver is further configured to receive mobile device data from one of the identified one or more mobile devices that specifies the power level of the corresponding one of the identified mobile devices, and

wherein the one or more processors are further configured to, when rendering the audio signals from the audio source, perform one or more of adjusting a volume of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device, cross-mixing the audio signals to be played by the corresponding one of the mobile devices with the audio signals to be played by one or more of the remaining mobile devices to accommodate the power level of the mobile device, and reducing at least some range of frequencies of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device.

22. The headend device of claim 12, wherein the audio source comprises one of a higher order ambisonic audio source data, a multi-channel audio source data and an object-based audio source data.

23. A headend device comprising:

means for identifying one or more mobile devices, each of the one or more mobile devices is separate from the headend device, includes a speaker, and is available to participate in a collaborative surround sound system;

means for configuring the collaborative surround sound system to utilize the speaker of each of the one or more mobile devices as one or more virtual speakers of the collaborative surround sound system;

means for receiving mobile device data from one of the identified one or more mobile devices that specifies a power level of the corresponding one of the identified mobile devices;

means for rendering audio signals from an audio source based on the determined power level of one of the mobile devices so as to control playback of the audio signals from the audio source to accommodate the power level of the identified mobile device and such that when the audio signals are played by the speakers of the one or more mobile devices the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system; and

means for transmitting the processed audio signals rendered from the audio source to each of the one or more mobile devices participating in the collaborative surround sound system.

24. The headend device of claim 23, wherein the one or more virtual speakers of the collaborative surround sound system appear to be placed in a location different than a location of at least one of the one or more mobile devices.

25. The headend device of claim 23, wherein the means for configuring the collaborative surround sound system comprises means for identifying speaker sectors at which each of the virtual speakers of the collaborative surround sound system are to appear to originate the audio playback of the audio signals, and

wherein the means for rendering the audio signals comprises means for rendering the audio signals from the audio source such that, when the audio signals are played by the speakers of the one or more mobile devices, the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system placed in a location within the corresponding identified speaker sectors.

26. The headend device of claim 23, further comprising means for receiving mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the means for configuring the collaborative surround sound system comprises means for configuring the collaborative surround sound system based on the associated mobile device data to utilize the speaker of each of the one or more mobile devices as the one or more virtual speakers of the collaborative surround sound system.

27. The headend device of claim 23, further comprising means for receiving mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein the means for configuring the collaborative surround sound system comprises:

means for determining that the one of the identified mobile devices is not in a specified location for playing the audio signals rendered from the audio source based on the location of the one of the identified mobile devices determined based on the mobile device data; and

means for prompting a user of the one of the identified mobile devices to re-position the one of the identified mobile devices to modify playback of the audio by the one of the identified mobile devices.

28. The headend device of claim 23, further comprising means for receiving mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein the means for rendering the audio signals comprises:

means for configuring an audio pre-processing function based on the location of one of the identified mobile devices so as to avoid prompting a user to move the one of the identified mobile devices; and

means for performing the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the location of the one of the identified mobile devices, and

wherein the means for transmitting the audio signals comprises means for transmitting at least the pre-processed portion of the audio signals rendered from the audio source to the one of the identified mobile devices.

29. The headend device of claim 23, further comprising means for receiving mobile device data from one of the identified one or more mobile devices that specifies one or more speaker characteristics of the speaker included within one of the identified mobile devices,

wherein the means for rendering the audio signals comprises:

means for configuring an audio pre-processing function by which to process the audio signals from the audio source based on the one or more speaker characteristics; and

means for performing the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the one or more speaker characteristics of the speaker included within the one of the identified mobile devices, and

wherein the means for transmitting the audio signals comprises means for transmitting at least the pre-processed portion of the audio signals to the one of the identified mobile devices.

30. The headend device of claim 23, further comprising means for receiving mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the mobile device data specifies one or more of a location of the corresponding one of the identified mobile devices, a frequency response of the speaker included within the corresponding one of the identified mobile devices, a maximum allowable sound reproduction level of the speaker included within the corresponding one of the identified mobile devices, the power level of the corresponding one of the identified mobile devices, a sync status of the corresponding one of the identified mobile devices, and a headphone status the corresponding one of the identified mobile devices.

31. The headend device of claim 23, further comprising means for determining that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals rendered from the audio source, wherein rendering the audio signals from the audio source comprises rendering the audio signals to reduce an amount of power required by the corresponding one of the mobile devices to play the audio signals based on the determination that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals.

32. The headend device of claim 23, further comprising means for receiving mobile device data from one of the identified one or more mobile devices that specifies the power level of the corresponding one of the identified mobile devices, and

wherein the means for rendering the audio signals from the audio source comprises one or more of:

means for adjusting a volume of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device;

means for cross-mixing the audio signals to be played by the corresponding one of the mobile devices with the audio signals to be played by one or more of the remaining mobile devices to accommodate the power level of the mobile device; and

means for reducing at least some range of frequencies of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device.

33. The headend device of claim 23, wherein the audio source comprises one of a higher order ambisonic audio source data, a multi-channel audio source data and an object-based audio source data.

34. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed cause one or more processors to:

identify one or more mobile devices, each of the one or more mobile devices is separate from the headend device, includes a speaker, and is available to participate in a collaborative surround sound system;

configure the collaborative surround sound system to utilize the speaker of each of the one or more mobile devices as one or more virtual speakers of the collaborative surround sound system;

receive mobile device data from one of the identified one or more mobile devices that specifies a power level of the corresponding one of the identified mobile devices;

render audio signals from an audio source based on the determined power level of one of the mobile devices so as to control playback of the audio signals from the audio source to accommodate the power level of the mobile device and such that when the audio signals are played by the speakers of the one or more mobile devices the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system; and

transmit the processed audio signals rendered from the audio source to each of the mobile device participating in the collaborative surround sound system.

35. The non-transitory computer-readable storage medium of claim 34, wherein the one or more virtual speakers of the collaborative surround sound system appear to be placed in a location different than a location of at least one of the one or more mobile devices.

36. The non-transitory computer-readable storage medium of claim 34,

wherein the instructions further cause, when executed, the one or more processors to, when configuring the collaborative surround sound system, identify speaker sectors at which each of the virtual speakers of the collaborative surround sound system are to appear to originate the audio playback of the audio signals, and

wherein the instructions further cause, when executed, the one or more processors to, when rendering the audio signals, render the audio signals from the audio source such that, when the audio signals are played by the speakers of the one or more mobile devices, the audio playback of the audio signals appears to originate from the one or more virtual speakers of the collaborative surround sound system placed in a location within the corresponding identified speaker sectors.

37. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the instructions further cause, when executed, the one or more processors to, when configuring the collaborative surround sound system, configure the collaborative surround sound system based on the associated mobile device data to utilize the speaker of each of the one or more mobile devices as the one or more virtual speakers of the collaborative surround sound system.

38. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices,

wherein the instructions further cause, when executed, the one or more processors to, when configuring the collaborative surround sound system, determine that the one of the identified mobile devices is not in a specified location for playing the audio signals rendered from the audio source based on the location of the one of the identified mobile devices determined based on the mobile device data, and prompt a user of the one of the identified mobile devices to re-position the one of the identified mobile devices to modify playback of the audio by the one of the identified mobile devices.

39. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from one of the identified one or more mobile devices that specifies a location of the one of the identified one or more mobile devices, wherein the instructions further cause, when executed, the one or more processors to, when rendering the audio signals, configure an audio pre-processing function based on the location of one of the identified mobile devices so as to avoid prompting a user to move the one of the identified mobile devices, and perform the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the location of the one of the identified mobile devices, and

wherein the instructions further cause, when executed, the one or more processors to, when transmitting the audio signals, transmit at least the pre-processed portion of the audio signals rendered from the audio source to the one of the identified mobile devices.

40. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from one of the identified one or more mobile devices that specifies one or more speaker characteristics of the speaker included within one of the identified mobile devices,

wherein the instructions further cause, when executed, the one or more processors to, when rendering the audio signals, configure an audio pre-processing function by which to process the audio signals from the audio source based on the one or more speaker characteristics, and perform the configured audio pre-processing function when rendering at least a portion of the audio signals from the audio source to control playback of the audio signals to accommodate the one or more speaker characteristics of the speaker included within the one of the identified mobile devices, and

wherein the instructions further cause, when executed, the one or more processors to, when transmitting the audio signals, transmit at least the pre-processed portion of the audio signals to the one of the identified mobile devices.

41. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from each of the identified one or more mobile devices that specifies aspects of the corresponding one of the identified mobile devices that impacts audio playback of the audio,

wherein the mobile device data specifies one or more of a location of the corresponding one of the identified mobile devices, a frequency response of the speaker included within the corresponding one of the identified mobile devices, a maximum allowable sound reproduction level of the speaker included within the corresponding one of the identified mobile devices, the power level of the corresponding one of the identified mobile devices, a sync status of the corresponding one of the identified mobile devices, and a headphone status the corresponding one of the identified mobile devices.

42. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to determine that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals rendered from the audio source, wherein rendering the audio signals from the audio source comprises rendering the audio signals to reduce an amount of power required by the corresponding one of the mobile devices to play the audio signals based on the determination that the power level of the corresponding one of the mobile devices is insufficient to complete playback of the audio signals.

43. The non-transitory computer-readable storage medium of claim 34, further comprising instructions that, when executed, cause the one or more processors to receive mobile device data from one of the identified one or more mobile devices that specifies the power level of the corresponding one of the identified mobile devices, and

wherein the instructions further cause, when executed, the one or more processors to, when rendering the audio signals from the audio source, perform one or more of:

adjusting a volume of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device;

cross-mixing the audio signals to be played by the corresponding one of the mobile devices with the audio signals to be played by one or more of the remaining mobile devices to accommodate the power level of the mobile device; and

reducing at least some range of frequencies of the audio signals to be played by the corresponding one of the mobile devices to accommodate the power level of the mobile device.

44. The non-transitory computer-readable storage medium of claim 34, wherein the audio source comprises one of a higher order ambisonic audio source data, a multi-channel audio source data and an object-based audio source data.