Synchronization of content between networked devices

• US 10,098,082 B2
• Filed: 12/16/2015
• Issued: 10/09/2018
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Keyser-Allen, Lee, Higdon, Thomas, Vega-Zayas, Luis, Lin, Ted, Sarkic, Andrej
• Primary Examiner(s): KASSIM, KHALED M
• Application Number:
  US14/971,796
• Publication Number:
  US 20170181113A1
• Time in Patent Office: 1,028 Days
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 21/4307 Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4392 involving audio buffer management
• CPC Class Codes: H04N 21/8113 comprising music, e.g. song in MP3 format
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04W 56/0015 one node acting as a reference for the others

Abstract

Networked devices are arranged to play content such as audio and video content in synchronization. A synching device receives a packet from an initiating device over a network. The packet may include an identifier of content to play, such as a URI, and an indication of when to play the content. A local clock time to play the content is determined based on a time of a local clock of the first device and the indication of when to play the content. The synching device obtains from a server the content based on the received identifier, and plays the content in synchronization with the initiating device based on a time on the local clock and the local clock time to play the content. In some embodiments, the synching device also adjusts a clock speed of the local clock reference to control rate of playback of the content.

Claims

1. A system comprising:

a first device comprising;

a local clock;

a network interface;

a processor and tangible, non-transitory computer-readable media having first instructions encoded therein and executable by the processor; and

a second device comprising;

a network interface, wherein the network interface of the first device is configured to communicate with the network interface of the second device over a network; and

a processor and tangible, non-transitory computer-readable media having second instructions encoded therein and executable by the processor of the second device;

wherein the second instructions when executed by the processor of the second device cause the second device to;

send an identifier of content to play and an indication of when to play the content to the first device over the network, wherein the second device subtracts a network latency from the indication of when to play the content;

wherein the first instructions when executed by the processor of the first device cause the first device to;

determine a local clock time to play the content, the local clock time to play the content based on a time of the local clock of the first device and the indication of when to play the content;

obtain, from a third device over the network, the content to play based on the received identifier of content to play; and

play the obtained content based on the time of the local clock and the local clock time to play the content,

wherein the content comprises a plurality of frames, and wherein the indication of when to play the content is a length of time until playback of a frame, the second instructions comprising instructions executable by the processor of the second device to send a synchronization packet comprising a new length of time until playback of the frame, and the first instructions comprising instructions executable by the processor of the first device to determine an expected length of time until playback of the frame based on a presentation timestamp embedded in the content, wherein the presentation timestamp indicates a timing of playback for the content, and adjust a clock speed of the local clock based on the expected length of time until playback of the frame and the new length of time until playback of the frame.

2. The system of claim 1, wherein the network latency is based on a mean of a round trip packet delay between the first device and the second device.

3. The system of claim 1, wherein the identifier of content to play is a uniform resource indicator (URI) which facilitates access to the content on the third device via the network.

4. The system of claim 1, wherein determining a local clock time to play the content comprises adding the length of time until playback starts to the time of the local clock.

5. The system of claim 1 wherein adjusting the clock speed comprises calculating the following expression:

(Delta E-DeltaR)/(Delta_R) where Delta_E is the expected length of time until playback of the frame and Delta_R is the new length of time until playback of the frame.

6. The system of claim 1, wherein sending the identifier of content to play comprises sending a packet with a sequence number.

7. The system of claim 6, wherein the first instructions further comprise instructions executable by the processor of the first device to send an acknowledgement packet to the second device, the acknowledgment packet including the sequence number of the packet.

8. A method for synchronizing playback between a first device and a second device, the method comprising:

receiving, by the first device from the second device over a network, an identifier of content to play and an indication of when to play the content, wherein a network latency is subtracted from the indication of when to play the content by the second device;

determining, by the first device, a local clock time to play the content, the local clock time to play the content based on a time of a local clock of the first device and the indication of when to play the content;

obtaining, by the first device from a third device over the network, the content to play based on the received identifier of content to play; and

playing, by the first device, the obtained content based on the time of the local clock and the local clock time to play the content,

wherein the content comprises a plurality of frames, and wherein the indication of when to play the content is a length of time until playback of a frame, the method further comprising receiving, by the first device from the second device over the network, a synchronization packet comprising a new length of time until playback of the frame, determining, by the first device, an expected length of time until playback of the frame based on a presentation timestamp embedded in the content, wherein the presentation timestamp indicates a timing of playback for the content, and adjusting, by the first device, a clock speed of the local clock based on the expected length of time until playback of the frame and the new length of time until playback of the frame.

9. The method of claim 8, wherein the first device is an audio playback device, wherein the content comprises audio content and video content, and wherein playing, by the first device, the content comprises dropping, by the first device, the video content and playing the audio content.

10. The method of claim 8, wherein the synchronization packet defines a new clock generation number.

11. The method of claim 8, further comprising playing, by the first device, the content in synchronization with the second device based on the presentation timestamp embedded in the content.

12. A tangible, non-transitory computer readable storage medium including instructions encoded therein and executable by a processor, wherein the instructions when executed cause the processor to implement a method comprising:

receiving, by the first device from the second device over a network, an identifier of content to play and an indication of when to play the content, wherein a network latency is subtracted from the indication of when to play the content by the second device;

determining, by the first device, a local clock time to play the content based on a time of a local clock of the first device and the indication of when to play the content;

obtaining, by the first device from a third device over the network, the content to play based on the received identifier of content to play; and

playing, by the first device, the obtained content based on the time of the local clock and the local clock time to play the content,

wherein the content comprises a plurality of frames, and wherein the indication of when to play the content is a length of time until playback of a frame, the method further comprising receiving, by the first device from the second device over the network, a synchronization packet which defines a new length of time until playback of the frame;

determining, by the first device, an expected length of time until playback of the frame based on a presentation timestamp embedded in the content, wherein the presentation timestamp indicates a timing of playback for the content; and

adjusting, by the first device, a clock speed of the local clock based on the expected length of time until playback of the frame and the new length of time until playback of the frame.