Method and apparatus for synchronizing playback of streaming media in multiple output devices

• US 20060149850A1
• Filed: 01/05/2005
• Published: 07/06/2006
• Est. Priority Date: 01/05/2005
• Status: Active Grant

• Current Assignee: Snap One LLC (Snap One Holdings Corp.)
• Original Assignee: Control4 Corporation (Snap One Holdings Corp.)
• Inventors: Bowman, Timothy D.
• Granted Patent: US 8,015,306 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4392 involving audio buffer management
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/4825 using a list of items to be played back in a given order, e.g. playlists
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]

Abstract

A method and apparatus for synchronizing streaming media with multiple output devices. One or more media servers serve media streams to one or more output devices (i.e., players). For playback synchronization, one output device is the “master”, whereas the remaining output devices are “slaves”. More data is requested from the media server by the “master” device to maintain a nominal buffer fill level over time. The “slave” devices receive streamed data from the media server at the rate determined by the master device'"'"'s data requests, and the average rate of data flow over the streaming network is thus controlled by the frequency of the single “master” device'"'"'s crystal. “Slave” devices make playback rate corrections to maintain respective buffer fill levels within upper and lower threshold levels. For slow networks, each media data packet timestamp is calculated from the time the master'"'"'s buffer reaches nominal level.

Claims

1. A media network comprising:

a media server configured to transmit streaming media over a network; and

a plurality of output devices configured to receive and play said streaming media, said plurality of output devices comprising;

a master device configured to request media data from said media server based on a master playback rate;

one or more slave devices each having a playback buffer with an associated buffer fill level;

wherein said one or more slave devices are configured to implement a playback rate correction when the respective buffer fill level is outside of a desired buffer fill range; and

wherein said media server is configured to transmit media data to said plurality of output devices when said master device requests said media data.

2. The media network of claim 1, wherein said media server comprises a memory comprising:

a plurality of device identifiers corresponding to said plurality of output devices;

an association between said plurality of device identifiers and said streaming media; and

a master flag associated with one of said plurality of device identifiers that corresponds to said master device.

3. The media network of claim 1, wherein said master device comprises a playback buffer into which said media data from said media server is written and from which said media data is consumed at said master playback rate.

4. The media network of claim 3, wherein said master is configured to request more media data from said media server when said playback buffer falls below a nominal level.

5. The media network of claim 4, wherein said master device is configured to record a time value corresponding to when said buffer reaches said nominal level, and to transmit said time value to said media server.

6. The media network of claim 5, wherein said media server transmits said media data in data packets, said data packets comprising a timestamp indicating when the respective packet should be played back.

7. The media network of claim 6, wherein said timestamp is calculated from said time value received from said master device.

8. The media network of claim 1, wherein each of said plurality of output devices is configured to:

detect one or more missing packets based on a timestamp of a current packet, and

write to its respective playback buffer a period of null data corresponding to said one or more missing packets.

9. The media network of claim 1, wherein each of said one or more slave devices comprises a crystal oscillator, and wherein one or more of said slave devices is configured to implement said playback rate correction by adjusting a clock rate of said crystal oscillator.

10. The media network of claim 1, wherein one or more of said slave devices is configured to implement said playback rate correction by adjusting a number of samples in said media data.

11. A method for synchronizing playback of streaming media in a network comprising a media server and a plurality of output devices, comprising:

selecting a master device from a plurality of output devices;

said master device sending a data request to a media server when a fill level of a master buffer falls below a desired level;

in response to said data request from said master device, said plurality of output devices receiving a data packet from said media server;

in one or more of said plurality of output devices other than said master device, determining that a fill level of a respective playback buffer is outside of a desired level range, and applying a playback rate adjustment corresponding to whether said fill level is above or below said desired level range.

12. The method of claim 11, further comprising said plurality of output devices obtaining a timestamp from said data packet, said timestamp indicating when said data packet should be played back.

13. The method of claim 12, further comprising:

one of said plurality of devices determining, based on said timestamp, that a prior data packet was skipped; and

said one of said plurality of output devices writing a period of null data to a respective playback buffer, said period corresponding to said prior data packet.

14. The method of claim 12, further comprising:

recording a time value corresponding to when said fill level of said master buffer reaches said desired level; and

using said time value to calculate said timestamp.

15. The method of claim 11, wherein applying said playback rate adjustment comprises controlling a clock source of a playback circuit.

16. The method of claim 11, wherein applying said playback rate adjustment comprises adjusting a number of samples written to said respective playback buffer.

17. The method of claim 11, further comprising:

upon receipt of a first packet of media data, one of said plurality of output devices determining that said respective playback buffer is empty; and

said one of said plurality of output devices writing a period of null data into said playback buffer ahead of said media data.

18. The method of claim 17, further comprising determining said period of null data as the time difference between a timestamp in said first data packet and a current time.

19. The method of claim 11, further comprising:

selecting said plurality of output devices from a registry of available output devices; and

associating said plurality of devices with a media stream.

20. The method of claim 11, wherein selecting said master device comprises:

sending a request packet to said plurality of output devices; and

flagging the first output device to respond to said request packet as said master device.

21. A playback device comprising:

a transceiver configured to receive data packets over a network;

a playback buffer, said playback buffer having a fill level;

a processor configured to obtain media data from said data packets and write said media data into said playback buffer;

a playback circuit configured to extract said media data from said buffer at a playback rate and provide a player output; and

a memory comprising;

a master flag configured to indicate whether said device is in a master mode or a slave mode;

computer code executable by said processor, said computer code comprising master mode instructions and slave mode instructions;

wherein said master mode instructions are configured to be executed when said master flag is set, and configured to cause said processor to perform at least the steps of;

waiting until said fill level is at a nominal level before writing media data of a current data packet to said playback buffer; and

requesting another data packet from a media server;

wherein said slave mode instructions are configured to be executed when said master flag is not set, and configured to cause said processor to perform at least the steps of;

determining whether said fill level of said playback buffer is within a desired range;

applying a playback rate correction if said fill level is not within said desired range; and

writing said media data of said current data packet to said playback buffer.

22. The device of claim 21, wherein said computer code further comprises instructions configured to cause said processor to perform at least the steps of:

obtaining a timestamp from said current data packet; and

based on said timestamp, determining whether a prior data packet was skipped; and

writing a period of null data to said playback buffer in an amount corresponding to a determined number of skipped data packets.

23. The device of claim 21, wherein said computer code further comprises instructions configured to cause said processor to perform at least the steps of:

obtaining a timestamp from said current data packet; and

writing a period of null data to an empty playback buffer in an amount calculated from said timestamp.

24. The device of claim 23, wherein said amount calculated from said timestamp comprises a difference between a value of said timestamp and a current time.

25. The device of claim 21, wherein said master mode instructions are further configured to cause said processor to perform at least the steps of:

recording a time value for when said fill level reaches said nominal level; and

sending said time value to a media server from which said current data packet was received, said time value providing a time basis for a timestamp of a subsequent data packet.

26. The device of claim 21, wherein said slave mode instructions configured to cause said processor to apply said playback rate correction comprise:

first correction instructions configured to cause said processor to perform at least the step of reducing a number of samples in said media data when said fill level is above said desired range; and

second correction instructions configured to cause said processor to perform at least the step of increasing the number of samples in said media data when said fill level is below said desired range.

27. The device of claim 21, wherein said slave mode instructions configured to cause said processor to apply said playback rate correction comprise:

correction instructions configured to cause said processor to perform at least the step of adjusting a control value for a clock source of said playback circuit to adjust said playback rate.