System and method for synchronizing media presentation at multiple recipients

• US 8,681,822 B2
• Filed: 04/04/2007
• Issued: 03/25/2014
• Est. Priority Date: 06/04/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bradley, Bob, Newberry, Robert Dale Jr.
• Primary Examiner(s): Rinehart, Mark
• Assistant Examiner(s): ANWAR, MOHAMMAD S
• Application Number:
  US11/696,679
• Publication Number:
  US 20070250761A1
• Time in Patent Office: 2,547 Days
• Field of Search: 370/503, 370/504, 370/505
• US Class Current: 370/503
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/1881 with schedule organisation, e.g. priority, sequence management
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A network media delivery system includes client devices and a host device. Each client device has a network interface, an engine for processing media data, and a media interface. The host device, which can be a computer, establishes network communication links with the client devices, which can be networked media stations, and sends media data to the client devices. The media data can be sent wirelessly as packets of media data transmitted at intervals to each client device. In one embodiment, the host device controls processing of media data such that processed media is delivered in a synchronized manner at each of the client devices. In another embodiment, the host device controls processing of media data such that processed media is delivered in a synchronized manner at the host device and at least one client device.

Claims

1. A client device, comprising:

a network interface for network communications;

a media interface for presenting processed media data;

a local clock; and

a processor coupled to the network interface, the media interface, and the local clock, wherein the processor is configured to;

receive a request from a host device for sending information about latency within the client device;

receive one or more media packets that are sent from the host device based on a packet timeline determined for the client device;

receive a playback timeline from the host device, the playback timeline indicative of when to play back, at the client device, media data included in the one or more media packets, wherein the playback timeline is determined by the host device based at least in part on the latency associated with the client device; and

play back media data included in each of the one or more media packets based on the playback timeline.

2. The client device of claim 1, wherein the processor is further configured to receive a unicast stream of first media packets including media data, each of the first media packets having a timestamp and a sequence number, the timestamps specifying when to process the media data.

3. The client device of claim 2, wherein the processor is further configured to synchronize the timestamp for each of the first media packets to the local clock prior to outputting the media data via the media interface.

4. The client device of claim 2, wherein the processor is configured to queue each of the first media packets for processing based on the sequence number of the media packet.

5. The client device of claim 4, wherein the processor is further configured to determine whether a media packet is missing from the queue based on a missing sequence number and send a request to the host device for retransmission of the missing media packet.

6. The client device of claim 1, wherein the processor is further configured to synchronize the local clock to an external reference clock.

7. The client device of claim 6, wherein to synchronize the local clock to the external reference clock, the processor is configured to send a request for time information and receive a response to the request, the response including a second packet containing time information to correlate the local clock to the external reference clock.

8. The client device of claim 1, wherein the latency within the client device includes a delay between the time when the media data is scheduled to be delivered to the client device and the time when the media data is presented by the client device.

9. A non-transitory computer readable medium including a plurality of instructions for controlling a processor in a client device to play back media data, the plurality of instructions comprising:

instructions that cause the processor to receive a request from a host device for sending information about latency information within the client device;

instructions that cause the processor to receive a unicast stream of media packets from the host device, each of the media packets in the unicast stream including a first timestamp, a portion of the media data, and a sequence number, wherein the unicast stream of media packets is sent from the host device to the client device using a packet timeline determined by the host device based at least in part on the latency information associated with the client device;

instructions that cause the processor to receive a playback timeline from the host device, the playback timeline determined by the host device and indicating when each portion of the media data is to be outputted at the client device; and

instructions that cause the processor to play back the media data based on the playback timeline.

10. The computer readable medium of claim 9 wherein the plurality of instructions further comprise:

instructions that cause the processor to receive a time announce packet different from the media packets from the host device, wherein the time announce packet includes a second timestamp that specifies timing relationship between the first timestamp and a reference clock of the host device; and

instructions that cause the processor to apply the timing relationship to the playback timeline when the media data play back reaches the time specified in the first timestamp.

11. The computer readable medium of claim 9, wherein the latency within the client device includes a delay between the time when the media data is scheduled to be delivered to the client device and the time when the media data is presented by the client device.

12. A method comprising:

receiving, by a client device, one or more media packets for play back, each of the one or more media packets including a first timestamp, a sequence number, and a portion of media data, wherein the one or more media packets are sent by a host device based on a packet timeline determined for the client device;

receiving, by the client device, a playback timeline indicating when each portion of the media data is to be outputted by the client device;

synchronizing, by the client device, a local clock of the client device to a reference clock of the host device;

receiving, by the client device, a time announcement packet, the time announcement packet including the first timestamp, a network time protocol (NTP) timestamp associated with the playback timeline, and a second timestamp indicative of when a time difference between the first timestamp and the NTP timestamp is to be applied to the playback timeline;

adjusting, by the client device, the playback timeline based on the second timestamp; and

outputting, by the client device, each portion of the media data based on the adjusted playback timeline;

wherein the time announcement packet is different from the one or more media packets.

13. The method of claim 12 the one or more media packets are received as a unicast stream.

14. The method of claim 12 further comprising:

receiving, by the client device, a request from the host device to communicate a latency associated with the client device; and

communicating, by the client device, the latency associated with the client device;

wherein the latency associated with the client device is used in determining the playback timeline.

15. The method of claim 12 wherein the time announcement packet is periodically received from the host device.

16. A method, comprising:

receiving, by a plurality of client devices, separate streams including a plurality of media packets, wherein each stream of media packets is transmitted from a host device based on a separate packet timeline determined by the host device for each of the plurality of client devices;

communicating, by each of the plurality of client devices to a host device, information about latency within the client device; and

receiving, by the plurality of client devices, a playback timeline indicative of when to play back, at the client device, media data associated with the plurality of media packets, wherein the playback timeline is determined by the host device based on the latency information associated for each of the plurality of client devices.

17. The method of claim 16, wherein the latency within the client device includes a delay between the time when the media data is scheduled to be delivered to the client device and the time when the media data is presented by the client device.