Synchronized audio mixing

• US 9,665,341 B2
• Filed: 02/09/2015
• Issued: 05/30/2017
• Est. Priority Date: 02/09/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Lin, Ted, Vega, Luis
• Primary Examiner(s): GAY, SONIA L
• Application Number:
  US14/617,604
• Publication Number:
  US 20160231981A1
• Time in Patent Office: 841 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04J 3/0664 unidirectional timestamps
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/439 Processing of audio elementary streams
• CPC Class Codes: H04N 21/4852 for modifying audio parameters, e.g. switching between mono and stereo
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A first example playback device includes one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the first playback device to perform functions. The functions include generating a mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream; and playing the mixed audio signal, where the one or more components of the second audio stream of the mixed audio signal played by the first playback device are played in synchrony with a second playback device configured to play the second audio stream. An example non-transitory computer-readable medium and an example method, both related to the first example playback device, are also disclosed herein.

Claims

1. A system, comprising:

a first playback device configured to play a first mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream, wherein the first audio stream is different from the second audio stream; and

a second playback device comprising;

one or more processors; and

tangible, non-transitory, computer-readable media storing instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising;

receiving, at the second playback device, the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating a second mixed audio signal comprising one or more of the components of the second audio stream and one or more components of the third audio stream; and

playing the second mixed audio signal, wherein the one or more components of the second audio stream of the first mixed audio signal played by the first playback device are played in synchrony with the one or more components of the second audio stream of the second mixed audio signal played by the second playback device.

2. The system of claim 1, wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

receiving, at the second playback device, first data representing the third audio stream and second data representing the second audio stream,

wherein generating the second mixed audio signal comprises using the first data and the second data to generate the second mixed audio signal.

3. The system of claim 1, wherein generating the second mixed audio signal comprises, for a given sample time of a set of sample times, determining a sum of (i) a first sample of the third audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.

4. The system of claim 3, wherein generating the second mixed audio signal further comprises:

determining that a particular sum corresponding to a particular sample time of the set of sample times is outside of a dynamic range of the second playback device; and

generating a sample of the second mixed audio signal that corresponds to the particular sample time, wherein the generated sample is within the dynamic range of the second playback device.

5. The system of claim 3, wherein generating the second mixed audio signal further comprises:

determining that a particular sum corresponding to a particular sample time of the set of sample times is outside of a dynamic range of the second playback device;

determining a scaling factor; and

for a given sample time of the set of sample times, scaling the sum corresponding to the given sample time by the scaling factor.

6. The system of claim 1, further comprising:

a third playback device,

wherein the tangible, non-transitory, computer-readable media on the second playback device further includes instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising;

receiving timing information from the third playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.

7. The system of claim 1, wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

receiving timing information from the first playback device,

wherein the timing information includes data indicative of a first time to initiate playback on the second playback device of at least a first component of the one or more of components of the second audio stream, and

wherein playing the second mixed audio signal comprises playing the second mixed audio signal at the first time based on the timing information.

8. The system of claim 1, wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

sending timing information to the first playback device, wherein the first playback device and the second playback device play the second audio stream in synchrony based on the timing information.

9. The system of claim 1, wherein the first audio stream corresponds to a first channel of multi-channel audio content and the second audio stream corresponds to a second channel of the multi-channel audio content.

10. The system of claim 1, wherein the first audio stream corresponds to a channel of multi-channel audio content and the second audio stream is monaural audio content.

11. The system of claim 1, wherein the second mixed audio signal comprises (i) a first portion of the third audio stream that fades out as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades in as the second mixed audio signal is played.

12. The system of claim 1, wherein the second mixed audio signal comprises (i) a portion of the third audio stream that fades in as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades out as the second mixed audio signal is played.

13. The system of claim 1, wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

playing the third audio stream;

receiving a command to fade out the third audio stream and fade in the second audio stream; and

generating the second mixed audio signal based on receiving the command, wherein the second mixed audio signal comprises (i) a portion of the third audio stream that fades out as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades in as the second mixed audio signal is played.

14. The system of claim 1, wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

playing the second audio stream;

receiving a command to fade out the second audio stream and fade in the third audio stream; and

generating the second mixed audio signal based on receiving the command, wherein the second mixed audio signal comprises (i) a portion of the third audio stream that fades in as the second mixed audio signal is played and (ii) a portion of the second audio stream that fades out as the second mixed audio signal is played.

15. The system of claim 1, wherein the tangible, non-transitory, computer-readable media on the second playback device is further configured to store a playback queue, wherein the second audio stream follows the third audio stream in the playback queue of the second playback device, and wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

playing the third audio stream;

determining that playback of the third audio stream will be completed within a threshold duration of time; and

generating the second mixed audio signal in response to determining that playback of the third audio stream will be completed within the threshold duration of time.

16. The system of claim 1, wherein the tangible, non-transitory, computer-readable media on the second playback device is further configured to store a playback queue, wherein the third audio stream follows the second audio stream in the playback queue, and wherein the instructions further include instructions that, when executed by the one or more processors, cause the second playback device to perform functions comprising:

playing the second audio stream;

determining that playback of the second audio stream will be completed within a threshold duration of time; and

generating the second mixed audio signal in response to determining that the second audio stream will be completed within the threshold duration of time.

17. A system comprising:

a first playback device comprising;

tangible, non-transitory, computer-readable first media storing instructions that, when executed by the first playback device, cause the first playback device to perform operations comprising;

receiving, at the first playback device, a first audio stream and a second audio stream, wherein the first audio stream is different from the second audio stream;

generating a first mixed audio signal comprising one or more components of the first audio stream and one or more components of the second audio stream; and

playing the first mixed audio signal; and

a second playback device comprising;

tangible, non-transitory, computer-readable second media storing instructions that, when executed by the second playback device, cause the second playback device to perform operations comprising;

receiving, at the second playback device, the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating a second mixed audio signal comprising one or more of the components of the second audio stream and one or more components of the third audio stream; and

playing the second mixed audio signal, wherein the one or more components of the second audio stream of the first mixed audio signal played by the first playback device are played in synchrony with the one or more components of the second audio stream of the second mixed audio signal played by the second playback device.

18. The system of claim 17, wherein the tangible, non-transitory computer-readable first media further includes instructions that, that when executed, cause the first playback device to perform operations comprising:

determining, for a given sample time of a set of sample times, a sum of (i) a first sample of the first audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.

19. A method comprising:

playing, via a first playback device, a first mixed audio signal comprising one or more components of a first audio stream and one or more components of a second audio stream, wherein the first audio stream is different from the second audio stream;

receiving, at a second playback device, the second audio stream and a third audio stream, wherein the third audio stream is different from the first audio stream and the second audio stream;

generating, at the second playback device, a second mixed audio signal comprising one or more of the one or more components of the second audio stream and one or more components of the third audio stream; and

playing, via the second playback device, the second mixed audio signal, wherein the one or more components of the second audio stream of the first mixed audio signal played by the first playback device are played in synchrony with the one or more components of the second audio stream played by the second playback device.

20. The method of claim 19, wherein generating the second mixed audio signal comprises, for a given sample time of a set of sample times, determining a sum of (i) a first sample of the first audio stream that corresponds to the given sample time and (ii) a second sample of the second audio stream that corresponds to the given sample time.