Synchronized audio mixing

• US 9,977,649 B2
• Filed: 05/28/2017
• Issued: 05/22/2018
• Est. Priority Date: 02/09/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Lin, Ted M., Vega, Luis
• Primary Examiner(s): Gay, Sonia
• Application Number:
  US15/607,507
• Publication Number:
  US 20170262257A1
• Time in Patent Office: 359 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04J 3/0664 unidirectional timestamps
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4852 for modifying audio parameters, e.g. switching between mono and stereo
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A first example playback device includes one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the first playback device to perform functions. The functions include generating a mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream; and playing the mixed audio signal, where the one or more components of the second audio stream of the mixed audio signal played by the first playback device are played in synchrony with a second playback device configured to play the second audio stream. An example non-transitory computer-readable medium and an example method, both related to the first example playback device, are also disclosed herein.

Claims

1. A method of operating a media playback system that includes a first playback device and a second playback device, the method comprising:

generating, at the first playback device, a first mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream, wherein the first audio stream is different from the second audio stream;

receiving, at the second playback device, the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating, at the second playback device, a second mixed audio signal comprising one or more of the components of the second audio stream and one or more components of the third audio stream; and

playing the first mixed audio signal and the second mixed audio signal from the first playback device and the second playback device, respectively, wherein the one or more components of the second audio stream of the first mixed audio signal played by the first playback device are played in synchrony with the one or more components of the second audio stream of the second mixed audio signal played by the second playback device.

2. The method of claim 1, wherein receiving, at the second playback device, the second audio stream and the third audio stream further comprises receiving, at the second playback device, first data representing the third audio stream and second data representing the second audio stream, and wherein generating the second mixed audio signal further comprises using the first data and the second data to generate the second mixed audio signal.

3. The method of claim 1, wherein generating the second mixed audio signal comprises, for a given sample time of a set of sample times, determining a sum of (i) a first sample of the third audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.

4. The method of claim 3, wherein generating the second mixed audio signal further comprises:

determining that a particular sum corresponding to a particular sample time of the set of sample times is outside of a dynamic range of the second playback device; and

generating a sample of the second mixed audio signal that corresponds to the particular sample time, wherein the generated sample is within the dynamic range of the second playback device.

5. The method of claim 3, wherein generating the second mixed audio signal further comprises:

determining that a particular sum corresponding to a particular sample time of the set of sample times is outside of a dynamic range of the second playback device;

determining a scaling factor; and

for a given sample time of the set of sample times, scaling the sum corresponding to the given sample time by the scaling factor.

6. The method of claim 1, wherein the media playback system further includes a third playback device, the method further comprising:

receiving timing information from the third playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.

7. The method of claim 1, further comprising:

receiving timing information from the first playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.

8. The method of claim 1, further comprising:

sending timing information to the first playback device, wherein the first playback device and the second playback device play the second audio stream in synchrony based on the timing information.

9. The method of claim 1, wherein playing the first mixed audio signal comprises playing the first audio stream, and playing the second mixed audio signal comprises playing the third audio stream, the method further comprising:

fading in a portion of the second audio stream in the first mixed audio signal and the second mixed audio signal.

10. The method of claim 1, wherein playing the first mixed audio signal comprises playing the first audio stream, and playing the second mixed audio signal comprises playing the third audio stream, the method further comprising:

fading out a portion of the second audio stream in the first mixed audio signal and the second mixed audio signal.

11. The method of claim 1, wherein playing the second mixed audio signal comprises:

fading out a portion of the third audio stream; and

fading in a portion of the second audio stream.

12. The method of claim 1, wherein playing the second mixed audio signal comprises:

fading in a portion of the third audio stream; and

fading out a portion of the second audio stream.

13. The method of claim 1, further comprising:

playing the third audio stream;

receiving a command to fade out the third audio stream and fade in the second audio stream; and

generating the second mixed audio signal based on receiving the command, wherein the second mixed audio signal comprises (i) a portion of the third audio stream that fades out as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades in as the second mixed audio signal is played.

14. The method of claim 1, further comprising:

playing the second audio stream;

receiving a command to fade out the second audio stream and fade in the third audio stream; and

generating the second mixed audio signal based on receiving the command, wherein the second mixed audio signal comprises (i) a portion of the third audio stream that fades in as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades out as the second mixed audio signal is played.

15. The method of claim 1, wherein the second audio stream follows the third audio stream in a playback queue of the second playback device, the method further comprising:

playing the third audio stream;

determining that playback of the third audio stream will be completed within a threshold duration of time; and

generating the second mixed audio signal in response to determining that playback of the third audio stream will be completed within the threshold duration of time.

16. The method of claim 1, wherein the third audio stream follows the second audio stream in a playback queue of the second playback device, the method further comprising:

playing the second audio stream;

determining that playback of the second audio stream will be completed within a threshold duration of time; and

generating the second mixed audio signal in response to determining that the second audio stream will be completed within the threshold duration of time.

17. A method of operating a media playback system that includes a first playback device and a second playback device, the method comprising:

playing a first mixed audio signal from the first playback device, wherein the first mixed audio signal comprises one or more components of a first audio stream and one or more components of a second audio stream, wherein the first audio stream is different from the second audio stream;

receiving, at the second playback device, the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating a second mixed audio signal comprising one or more of the components of the second audio stream and one or more components of the third audio stream; and

playing the second mixed audio signal, wherein the one or more components of the second audio stream of the first mixed audio signal played by the first playback device are played in synchrony with the one or more components of the second audio stream of the second mixed audio signal played by the second playback device.

18. The method of claim 17, further comprising:

determining, for a given sample time of a set of sample times, a sum of (i) a first sample of the first audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.

19. The method of claim 17, wherein playing the first mixed audio signal comprises playing the first audio stream, and playing the second mixed audio signal comprises playing the third audio stream, the method further comprising:

fading in, at a first time, a first portion of the second audio stream in the first mixed audio signal and the second mixed audio signal; and

fading out, at a second time, a second portion of the second audio stream in the first mixed audio signal and the second mixed audio signal, wherein the second time is later than the first time.

20. A media playback system, comprising:

a first playback device that includes a tangible, non-transitory first memory storing instructions that, when executed by one or more processors of the first playback device, cause the first playback device to perform operations comprising;

generating a first mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream, wherein the first audio stream is different from the second audio stream; and

playing the first mixed audio signal; and

a second playback device that includes a tangible, non-transitory second memory storing instructions that, when executed by one or more processors of the second playback device, cause the second playback device to perform operations comprising;

receiving the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating a second mixed audio signal comprising one or more of the components of the second audio stream and one or more components of a third audio stream; and

playing the second mixed audio signal, wherein the one or more components of the second audio stream of the second mixed audio signal played by the second playback device are played in synchrony with the one or more components of the second audio stream of the first mixed audio signal played by the first playback device.

21. The media playback system of claim 20, wherein receiving, at the second playback device, the second audio stream and the third audio stream further comprises receiving, at the second playback device, first data representing the third audio stream and second data representing the second audio stream, and wherein generating the second mixed audio signal further comprises using the first data and the second data to generate the second mixed audio signal.

22. The media playback system of claim 20, wherein generating the second mixed audio signal comprises, for a given sample time of a set of sample times, determining a sum of (i) a first sample of the third audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.

23. The media playback system of claim 20, wherein the media playback system further includes a third playback device, and wherein the operations performed by the second playback device further comprise:

receiving timing information from the third playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.

24. The media playback system of claim 20, wherein the operations performed by the second playback device further comprise:

receiving timing information from the first playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.