Synchronized audio mixing

• US 10,387,110 B2
• Filed: 03/26/2018
• Issued: 08/20/2019
• Est. Priority Date: 02/09/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sohos Incorporated
• Inventors: Lin, Ted M., Vega, Luis
• Primary Examiner(s): Gay, Sonia L
• Application Number:
  US15/935,966
• Publication Number:
  US 20180285060A1
• Time in Patent Office: 512 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04J 3/0664 unidirectional timestamps
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4852 for modifying audio parameters, e.g. switching between mono and stereo
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A first example playback device includes one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the first playback device to perform functions. The functions include generating a mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream; and playing the mixed audio signal, where the one or more components of the second audio stream of the mixed audio signal played by the first playback device are played in synchrony with a second playback device configured to play the second audio stream. An example non-transitory computer-readable medium and an example method, both related to the first example playback device, are also disclosed herein.

Claims

1. A method of operating a media playback system comprising a first computing device, a second computing device, and a third computing device, the method comprising:

at the first computing device, generating a first mixed audio signal comprising a first audio stream received from a first audio source and a third audio stream received from a third audio source, wherein the first audio stream is different from the third audio stream;

at the second computing device, generating a second mixed audio signal comprising a second audio stream received from a second audio source and the third audio stream received from the third audio source, wherein the second audio stream is different than the first audio stream and the third audio stream;

at the first computing device, playing the first mixed audio signal, wherein playing the first mixed audio signal comprises playing the third audio stream in synchrony with playback of the third audio stream by the second computing device based on playback timing for the third audio stream received from the third computing device; and

at the second computing device, playing the second mixed audio signal, wherein playing the second mixed audio signal comprises playing the third audio stream in synchrony with playback of the third audio stream by the first computing device based on playback timing for the third audio stream received from the third computing device.

2. The method of claim 1, wherein the first audio source comprises at least one of a first uniform resource identifier (URI) or uniform resource locator (URL), and wherein the second audio source comprises at least one of a second URI or URL.

3. The method of claim 2, wherein the third computing device comprises the third audio source.

4. The method of claim 1, wherein generating the first mixed audio signal comprises, at a first sample time, determining a sum of (i) a sample of the first audio stream at the first sample time and (ii) a sample of the third audio stream at the first sample time.

5. The method of claim 4, wherein generating the first mixed audio signal further comprises:

determining, for the first sample time, that the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time would be outside of a dynamic range of the first computing device; and

in response determining, for the first sample time that the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time would be outside of the dynamic range of the first computing device, generating a sample of the first mixed audio signal at the first sample time that is within the dynamic range of the first computing device.

6. The method of claim 5, wherein generating the sample of the first mixed audio signal at the first sample time that is within the dynamic range of the first computing device comprises scaling the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time by a scaling factor.

7. The method of claim 1 further comprising:

at the first computing device, receiving the playback timing for the third audio stream from the third computing device, wherein the playback timing comprises data indicative of a first time to initiate playback on the first computing device of at least a first component of the third audio stream, and

at the second computing device, receiving the playback timing for the third audio stream from the third computing device, wherein the playback timing comprises data indicative of the first time to initiate playback on the second computing device of at least the first component of the third audio stream.

8. The method of claim 1, further comprising:

sending timing information to the first computing device and the second computing device, wherein the first computing device and the second computing device use the timing information to play the third audio stream in synchrony with each other.

9. The method of claim 1, wherein playing the first mixed audio signal at the first computing device comprises fading in a portion of the third audio stream into the first mixed audio signal, and wherein playing the second mixed audio signal at the second computing device comprises fading in the portion of the third audio stream into the second mixed audio signal.

10. The method of claim 1, wherein playing the first mixed audio signal at the first computing device comprises fading out a portion of the third audio stream from the first mixed audio signal, and wherein playing the second mixed audio signal at the second computing device comprises fading out the portion of the third audio stream from the second mixed audio signal.

11. The method of claim 1, wherein playing the first mixed audio signal at the first computing device comprises (i) fading out a portion of the first audio stream from the first mixed audio signal and (ii) fading in a portion of the third audio stream into the first mixed audio signal.

12. The method of claim 1, wherein playing the second mixed audio signal at the second computing device comprises (i) fading out a portion of the second audio stream from the second mixed audio signal and (ii) fading in a portion of the third audio stream into the second mixed audio signal.

13. The method of claim 1, further comprising:

receiving a command at the first computing device to fade out the first audio stream and fade in the third audio stream; and

in response to receiving the command at the first computing device to fade out the first audio stream and fade in the third audio stream, generating the first mixed audio signal comprising (i) a portion of the first audio stream that fades out as the first mixed audio signal is played and (ii) a portion of the third audio stream that fades in as the first mixed audio signal is played.

14. The method of claim 1, further comprising:

receiving a command at the second computing device to fade out the second audio stream and fade in the third audio stream; and

in response to receiving the command at the second computing device to fade out the second audio stream and fade in the third audio stream, generating the second mixed audio signal comprising (i) a portion of the second audio stream that fades out as the second mixed audio signal is played and (ii) a portion of the third audio stream that fades in as the second mixed audio signal is played.

15. Tangible, non-transitory computer-readable media comprising instructions encoded thereon, wherein the instructions, when executed by one or more processors, cause a media playback system comprising a first computing device, a second computing device, and a third computing device, to perform a method comprising:

at the first computing device, generating a first mixed audio signal comprising a first audio stream received from a first audio source and a third audio stream received from a third audio source, wherein the first audio stream is different from the third audio stream;

at the second computing device, generating a second mixed audio signal comprising a second audio stream received from a second audio source and the third audio stream received from the third audio source, wherein the second audio stream is different than the first audio stream and the third audio stream;

at the first computing device, playing the first mixed audio signal, wherein playing the first mixed audio signal comprises playing the third audio stream in synchrony with playback of the third audio stream by the second computing device based on playback timing for the third audio stream received from the third computing device; and

at the second computing device, playing the second mixed audio signal, wherein playing the second mixed audio signal comprises playing the third audio stream in synchrony with playback of the third audio stream by the first computing device based on playback timing for the third audio stream received from the third computing device.

16. The tangible, non-transitory computer-readable media of claim 15, wherein the first audio source comprises at least one of a first uniform resource identifier (URI) or uniform resource locator (URL), and wherein the second audio source comprises at least one of a second URI or URL.

17. The tangible, non-transitory computer-readable media of claim 16, wherein the third computing device comprises the third audio source.

18. The tangible, non-transitory computer-readable media of claim 15, wherein generating the first mixed audio signal comprises, at a first sample time, determining a sum of (i) a sample of the first audio stream at the first sample time and (ii) a sample of the third audio stream at the first sample time.

19. The tangible, non-transitory computer-readable media of claim 18, wherein generating the first mixed audio signal further comprises:

determining, for the first sample time, that the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time would be outside of a dynamic range of the first computing device; and

in response determining, for the first sample time that the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time would be outside of the dynamic range of the first computing device, generating a sample of the first mixed audio signal at the first sample time that is within the dynamic range of the first computing device.

20. The tangible, non-transitory computer-readable media of claim 19, wherein generating the sample of the first mixed audio signal at the first sample time that is within the dynamic range of the first computing device comprises scaling the sum of the sample of the first audio stream at the first sample time and the sample of the third audio stream at the first sample time by a scaling factor.

21. The tangible, non-transitory computer-readable media of claim 15, wherein the method further comprises:

at the first computing device, receiving the playback timing for the third audio stream from the third computing device, wherein the playback timing comprises data indicative of a first time to initiate playback on the first computing device of at least a first component of the third audio stream, and

at the second computing device, receiving the playback timing for the third audio stream from the third computing device, wherein the playback timing comprises data indicative of the first time to initiate playback on the second computing device of at least the first component of the third audio stream.

22. The tangible, non-transitory computer-readable media of claim 15, wherein the method further comprises:

sending timing information to the first computing device and the second computing device, wherein the first computing device and the second computing device use the timing information to play the third audio stream in synchrony with each other.

23. The tangible, non-transitory computer-readable media of claim 15, wherein playing the first mixed audio signal at the first computing device comprises fading in a portion of the third audio stream into the first mixed audio signal, and wherein playing the second mixed audio signal at the second computing device comprises fading in the portion of the third audio stream into the second mixed audio signal.

24. The tangible, non-transitory computer-readable media of claim 15, wherein playing the first mixed audio signal at the first computing device comprises fading out a portion of the third audio stream from the first mixed audio signal, and wherein playing the second mixed audio signal at the second computing device comprises fading out the portion of the third audio stream from the second mixed audio signal.

25. The tangible, non-transitory computer-readable media of claim 15, wherein playing the first mixed audio signal at the first computing device comprises (i) fading out a portion of the first audio stream from the first mixed audio signal and (ii) fading in a portion of the third audio stream into the first mixed audio signal.

26. The tangible, non-transitory computer-readable media of claim 15, wherein playing the second mixed audio signal at the second computing device comprises (i) fading out a portion of the second audio stream from the second mixed audio signal and (ii) fading in a portion of the third audio stream into the second mixed audio signal.

27. The tangible, non-transitory computer-readable media of claim 15, further comprising:

receiving a command at the first computing device to fade out the first audio stream and fade in the third audio stream; and

in response to receiving the command at the first computing device to fade out the first audio stream and fade in the third audio stream, generating the first mixed audio signal comprising (i) a portion of the first audio stream that fades out as the first mixed audio signal is played and (ii) a portion of the third audio stream that fades in as the first mixed audio signal is played.

28. The tangible, non-transitory computer-readable media of claim 15, further comprising:

receiving a command at the second computing device to fade out the second audio stream and fade in the third audio stream; and

in response to receiving the command at the second computing device to fade out the second audio stream and fade in the third audio stream, generating the second mixed audio signal comprising (i) a portion of the second audio stream that fades out as the second mixed audio signal is played and (ii) a portion of the third audio stream that fades in as the second mixed audio signal is played.