Synchronization of content between networked devices

• US 10,575,270 B2
• Filed: 10/08/2018
• Issued: 02/25/2020
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Keyser-Allen, Lee, Higdon, Thomas, Vega-Zayas, Luis, Lin, Ted, Sarkic, Andrej
• Primary Examiner(s): Kassim, Khaled M
• Application Number:
  US16/154,685
• Publication Number:
  US 20190045471A1
• Time in Patent Office: 505 Days
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 21/4307 Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4392 involving audio buffer management
• CPC Class Codes: H04N 21/8113 comprising music, e.g. song in MP3 format
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04W 56/0015 one node acting as a reference for the others

Abstract

Networked devices are arranged to play content such as audio and video content in synchronization. A synching device receives a packet from an initiating device over a network. The packet may include an identifier of content to play, such as a URI, and an indication of when to play the content. A local clock time to play the content is determined based on a time of a local clock of the first device and the indication of when to play the content. The synching device obtains from a server the content based on the received identifier, and plays the content in synchronization with the initiating device based on a time on the local clock and the local clock time to play the content. In some embodiments, the synching device also adjusts a clock speed of the local clock reference to control rate of playback of the content.

Claims

1. A first playback device comprising:

a processor;

a non-transitory computer-readable medium;

a network interface configured to communicate over a network; and

program instructions stored on the non-transitory computer-readable medium that, when executed by the processor, cause the first playback device to perform functions comprising;

receiving first content from a source device, wherein the first content is associated with second content to be played back by a second playback device, wherein the second content comprises a plurality of frames;

determining an initial indication of when to play the second content, wherein the initial indication of when to play the second content is a length of time until playback of a first frame in the plurality of frames;

determining a network latency of the network;

updating the indication of when to play the second content by subtracting the network latency from the length of time until playback of the first frame in the plurality of frames;

based on the updated indication, playing back the first content in synchrony with the second content being played back by the second playback device; and

while playing back the first content in synchrony with playback of the second content by the second playback device, (i) receiving a synchronization packet that indicates a new length of time until playback of a second frame of the plurality of frames and (ii) further updating the indication of when to play the second content based on the new length of time to maintain synchronous playback of the first and second content.

2. The first playback device of claim 1, wherein determining the network latency of the network comprises:

sending a test packet to the second playback device at a first time based on a local clock of the first playback device;

receiving an acknowledgement packet from the second playback device at a second time based on the local clock of the first playback device;

determining a test packet delay calculation based on a difference between the second time and the first time; and

determining the network latency based on the test packet delay calculation.

3. The first playback device of claim 1, wherein determining the network latency of the network comprises:

sending a plurality of test packets to the second playback device, wherein each of the test packets is sent to the second playback device at a different time based on a local clock of the first playback device;

receiving a plurality of acknowledgment packets in response to the plurality of test packets;

determining a plurality of test packet delays based on a difference of time between the local clock of the first playback device at a time of sending a given test packet and at a time of receiving a corresponding acknowledgment packet; and

determining the network latency based on a mean of the plurality of test packet delays.

4. The first playback device of claim 1, the functions further comprising sending an identifier of the second content, the identifier of the second content comprising a uniform resource indicator (URI), and wherein the identifier of the first content comprises the URI.

5. The first playback device of claim 1, wherein the indication of when to play the second content includes a presentation timestamp indicating a place in the second content where playback is to be initiated.

6. The first playback device of claim 1, further comprising a user interface, wherein the functions comprise receiving, via the user interface, a user input associated with at least one of the first content or the second content.

7. The first playback device of claim 1, wherein the functions comprise receiving, via a controller connected to the first playback device by the network, a user input associated with at least one of the first content or the second content.

8. The first playback device of claim 1, wherein the second content comprises video content, and wherein the first content comprises audio content associated with the video content.

9. The first playback device of claim 8, wherein receiving the first content comprises receiving the video content combined with the audio content, and wherein the functions comprise dropping the video content from the first content.

10. The first playback device of claim 1, wherein receiving the first content comprises receiving the first content from the source device over the network, and wherein the second playback device is configured to receive the second content from the source device over the network.

11. A method, comprising:

receiving first content from a source device, wherein the first content is associated with second content to be played back by a second playback device, wherein the second content comprises a plurality of frames;

determining an initial indication of when to play the second content, wherein the initial indication of when to play the second content is a length of time until playback of a first frame in the plurality of frames;

determining a network latency of the network;

updating the indication of when to play the second content by subtracting the network latency from the length of time until playback of the first frame in the plurality of frames;

based on the updated indication, playing back the first content in synchrony with playback of the second content by the second playback device; and

while playing back the first content in synchrony with playback of the second content by the second playback device, (i) receiving a synchronization packet that indicates a new length of time until playback of a second frame of the plurality of frames and (ii) further updating the indication of when to play the second content based on the new length of time to maintain synchronous playback of the first and second content by the respective first and second playback devices.

12. The method of claim 11, wherein determining the network latency of the network comprises:

sending a test packet to the second playback device at a first time based on a local clock of the first playback device;

receiving an acknowledgement packet from the second playback device at a second time based on the local clock of the first playback device;

determining a test packet delay calculation based on a difference between the second time and the first time; and

determining the network latency based on the test packet delay calculation.

13. The method of claim 11, wherein determining the network latency of the network comprises:

sending a plurality of test packets to the second playback device, wherein each of the test packets is sent to the second playback device at a different time based on a local clock of the first playback device;

receiving a plurality of acknowledgment packets in response to the plurality of test packets;

determining a plurality of test packet delays based on a difference of time between the local clock of the first playback device at a time of sending a given test packet and at a time of receiving a corresponding acknowledgment packet; and

determining the network latency based on a mean of the plurality of test packet delays.

14. The method of claim 11, further comprising sending an identifier of the second content, the identifier of the second content comprising a uniform resource indicator (URI), and wherein the identifier of the first content comprises the URI.

15. The method of claim 11, wherein the indication of when to play the second content includes a presentation timestamp indicating a place in the second content where playback is to be initiated.

16. The method of claim 11, further comprising receiving, via a user interface, a user input associated with at least one of the first content or the second content.

17. The method of claim 11, further comprising receiving, via a controller connected to the first playback device by the network, a user input associated with at least one of the first content or the second content.

18. The method of claim 11, wherein the second content comprises video content, and wherein the first content comprises audio content associated with the video content.

19. The method of claim 18 wherein receiving the first content comprises receiving the video content combined with the audio content, and wherein the method further comprises dropping the video content from the first content.

20. The method of claim 18, wherein receiving the first content comprises receiving the first content from the source device over the network, and wherein the second playback device is configured to receive the second content from the source device over the network.