System and Method for Synchronizing Media Presentation at Multiple Recipients

• US 20070110074A1
• Filed: 01/02/2006
• Published: 05/17/2007
• Est. Priority Date: 06/04/2004
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Newberry, Robert, Bradley, Bob
• Application Number:
  US11/306,557
• Publication Number:
  US 20070110074A1
• Time in Patent Office: Days
• US Class Current: 370/395.510
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 12/1881 with schedule organisation, e.g. priority, sequence management
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A network media delivery system includes client devices and a host device. Each client device has a network interface, an engine for processing media data, and a media interface. The host device, which can be a computer, establishes network communication links with the client devices, which can be networked media stations, and sends media data to the client devices. The media data can be sent wirelessly as packets of media data transmitted at intervals to each client device. In one embodiment, the host device controls processing of media data such that processed media is delivered in a synchronized manner at each of the client devices. In another embodiment, the host device controls processing of media data such that processed media is delivered in a synchronized manner at the host device and at least one client device.

Claims

1. A host device, comprising:

a network interface for network communication; and

a reference clock,

wherein the host device is configured to;

establish a first network communication link with a first client device;

send media data to the first client device via the first network communication link, the media data having a presentation timeline based on the reference clock, the presentation timeline specifying when to present the media data;

synchronize a local clock of the first client device with the reference clock; and

control processing of media data at the first client device such that the first client device presents processed media according to the presentation timeline.

2. The host device of claim 1, wherein the host device comprises a personal computer.

3. The host device of claim 1, wherein to send media data to the first client device via the first network communication link, the host device is configured to send a unicast stream of packets containing media data to the first client device, each packet having a timestamp at which to present the media data of the packet.

4. The host device of claim 3, wherein the packets comprise Real-Time Transport Protocol encapsulated in User Datagram Protocol packets.

5. The host device of claim 3, wherein the host device is configured to send the packets containing media data at substantially regular intervals.

6. The host device of claim 5, wherein the substantially regular intervals have a fixed delay based on a processing rate used by an engine of the first client device.

7. The host device of claim 1, wherein to synchronize the local clock of the first client device with the reference clock, the host device is configured to send packets to the first client device in response to requests from the first client device, the packets containing time information for correlating the local clock with the reference clock.

8. The host device of claim 7, wherein the packets comprise Network Time Protocol (NTP) encapsulated in Real-Time Transport Control Protocol (RTCP) packets.

9. The host device of claim 8, wherein the packets contain a Real-Time Transport Protocol (RTP) timestamp at which the request form the host device is transmitted, a first NTP timestamp at which the request from the first client device originated, a second NTP timestamp at which the request from the first client device is received by the host device, and a third NTP timestamp at which the response from the host device is transmitted.

10. The host device of claim 1, wherein to control processing of media data, the host device is configured to adjust a presentation time of processed media at the first client device based on a maximum latency value determined for the first client device.

11. The host device of claim 1, wherein the host device is configured to:

establish a second network communication link with a second client device;

send media data to the second client device via the second network communication link, the media data having a presentation timeline based on the reference clock, the presentation timeline specifying when to present the media data;

synchronize a local clock of the second client device with the reference clock; and

control processing of media data at the second client device such that the second client device presents processed media in a synchronized manner commensurate with the first client device.

12. The host device of claim 1, wherein the host device further comprises:

an engine for processing the media data into processed media; and

a media interface for presenting processed media,

wherein the host device is further configured to control processing of media data at the host device such that the host device presents processed media in a synchronized manner commensurate with the first client device.

13. A host device, comprising:

a network interface for network communication,

wherein the host device is configured to;

establish a network communication link with each of a plurality of client devices;

send media data to each of the client devices via the network communication links; and

control processing of media data at each of the client devices such that processed media is presented in a synchronized manner at each of the client devices.

14. The host device of claim 13, wherein the host device comprises a personal computer.

15. The host device of claim 13, wherein to send media data to each of the client devices via the network communication links, the host device is configured to send a unicast stream of packets containing media data to each of the client devices, each packet having a timestamp at which to present the media data of the packet.

16. The host device of claim 15, wherein the packets comprise Real-Time Transport Protocol encapsulated in User Datagram Protocol packets.

17. The host device of claim 15, wherein the host device is configured to send the packets containing media data at substantially regular intervals.

18. The host device of claim 17, wherein the substantially regular intervals have a fixed delay based on a processing rate used by the engines of the client devices.

19. The host device of claim 13, wherein:

the host device comprises a reference clock,

each of the client devices comprises a local clock, and to control processing of media data, the host device is configured to synchronize each of the local clocks with the reference clock.

20. The host device of claim 19, wherein to synchronize each of the local clocks with the reference clock, the host device is configured to send packets to each of the client devices in response to requests from each of the client devices, the packets containing time information for correlating the local clocks with the reference clock.

21. The host device of claim 20, wherein the packets comprise Network Time Protocol (NTP) encapsulated in Real-Time Transport Control Protocol (RTCP) packets.

22. The host device of claim 21, wherein the packets contain a Real-Time Transport Protocol (RTP) timestamp at which the request form the host device is transmitted, a first NTP timestamp at which the request from the client device originated, a second NTP timestamp at which the request from the client device is received by the host device, and a third NTP timestamp at which the response from the host device is transmitted.

23. The host device of claim 13, wherein to control processing of media data, the host device is configured to adjust a presentation time of processed media at each of the client devices based on a maximum latency value determined for the client devices.

24. The host device of claim 13, wherein the host device further comprises:

an engine for processing the media data into processed media; and

a media interface for presenting processed media,

wherein the host device is further configured to control processing of media data at the host device such that the host device presents processed media in a synchronized manner commensurate with the plurality of client devices.

25. A client device, comprising:

a network interface for network communication;

an engine for processing media data into processed media; and

a media interface for presenting processed media,

wherein the client device is configured to;

receive media data via a network communication link;

receive synchronized timing information via the network communication link; and

process media data with the engine such that the client device presents processed media according to the synchronized timing information received.

26. The client device of claim 25, wherein the network interface of the client device comprises a wireless network interface.

27. The client device of claim 25, wherein the client device comprises an integrated wireless network access point having a power adapter, a wireless network interface, a wired network interface, and a multimedia interface encapsulated within a single integrated casing.

28. The client device of claim 25, wherein the client device comprises an entertainment device connected thereto for presenting processed media, wherein the entertainment device is selected from the group consisting of an amplifier, a powered speaker, a stereo system, a video cassette recorder, a DVD player, a television, home theatre system, a laptop, and a personal computer.

29. The client device of claim 25, wherein to receive media data, the client device is configured to receive a unicast stream of packets containing media data, each packet having a timestamp and a sequence number.

30. The client device of claim 29, wherein the packets comprise Real-Time Transport Protocol encapsulated in User Datagram Protocol packets.

31. The client device of claim 29, wherein the client device is configured to queue packets for processing by the engine based on the sequence numbers.

32. The client device of claim 29, wherein the client device is configured to:

determine whether a packet is missing based on the sequence numbers; and

send a request for retransmission of a packet determined missing.

33. The client device of claim 25, wherein the client device comprises a local clock, and wherein to process media data with the engine according to the synchronized timing information, the client device is configured to synchronize the local clock based on the synchronized timing information.

34. The client device of claim 33, wherein to synchronize the local clock based on the synchronized timing information, the client device is configured to:

send requests for synchronized timing information; and

receive responses to the requests, the responses having packets containing time information for correlating the local clock with a reference clock.

35. The client device of claim 34, wherein the packets comprise Network Time Protocol (NTP) encapsulated in Real-Time Transport Control Protocol (RTCP) packets.

36. The client device of claim 35, wherein the packets contain a Real-Time Transport Protocol (RTP) timestamp at which the request form the host device is transmitted, a first NTP timestamp at which the request from the client device originated, a second NTP timestamp at which the request from the client device is received by the host device, and a third NTP timestamp at which the response from the host device is transmitted.

37. The client device of claim 33, wherein to present the processed media in a synchronized manner, the client device is configured to:

construct media data in received packets into a presentation timeline; and

process media data in the presentation timeline with the engine based on timestamps in the packets correlated to the synchronized time of the local clock of the client device.

38. A network media system, comprising:

means for transferring media data between a plurality of devices;

means for transferring time information between the devices;

means for synchronizing clocks at each of the devices based on the time information;

means for processing transferred media data into processed media at each of the devices; and

means for presenting processed media in a synchronized manner at each of the devices.

39. A network media presentation method, comprising the steps of:

identifying at least one client device connected to a network, the at least one client device capable of processing media data into processed media;

establishing a communication link with the at least one client device via the network;

sending media data to the at least one client device via the communication link;

processing media data into processed media at the at least client device; and

controlling processing of media data at the at least one client device such that processed media is presented in a synchronized manner commensurate with at least one other device.

40. The method of claim 39, wherein the step of identifying comprises receiving user input selecting which identified devices are designated to be a destination for presenting media.

41. The method of claim 39, wherein the step of establishing the communication link comprises establishing the communication link between a host device and the at least one client device.

42. The method of claim 39, wherein the step of sending media data comprises sending a unicast stream of packets containing media data to the at least one client device, each packet having a timestamp and a sequence number.

43. The method of claim 42, wherein the step of processing media data comprises queuing packets for processing by an engine at the at least one client device based on the sequence numbers.

44. The method of claim 42, wherein the step of processing media data comprises:

determining at the at least one client device whether a packet is missing based on the sequence numbers; and

sending a request for retransmission of a packet determined missing.

45. The method of claim 44, wherein the step of sending media data further comprises sending a response to the request, the response containing the packet determined missing.

46. The method of claim 42, wherein the step of sending media data comprises sending the packets containing media data at substantially regular intervals.

47. The method of claim 39, wherein:

each of the devices comprises a clock, and

the step of control processing of media data comprises synchronizing each of the clocks with a reference clock.

48. The method of claim 47, wherein the step of synchronizing comprises sending packets to each of the devices, the packets containing time information for correlating the local clocks with the reference clock.

49. The method of claim 47, wherein the step of processing media data comprises the steps of:

constructing media data contained in received packets into a presentation timeline; and

processing media data in the presentation timeline based on timestamps in the packets correlated to the synchronized time of the local clock of the device.

50. The method of claim 39, wherein the step of controlling processing of media data comprises adjusting a presentation time of processed media at each of the devices based on a maximum latency value determined for the devices.

51. The method of claim 39, wherein the step of controlling processing of media data comprises:

determining a number of gaps in media occurring at each of the devices; and

disabling presentation of processed media or retransmission of lost packets of media data at a given device based on the number of gaps determined for that device.

52. An electronically readable medium having instructions encoded thereon executable by a device for performing a network media presentation method, the method comprising the steps of:

identifying at least one client device connected to a network, the at least one client device capable of processing media data into processed media;

establishing a communication link with the at least one client device via the network;

sending media data to the at least one client device via the communication link;

processing media data into processed media at the at least client device; and

controlling processing of media data at the at least one client device such that processed media is presented in a synchronized manner commensurate with at least one other device.

53. The medium of claim 52, wherein the step of identifying comprises receiving user input selecting which identified devices are designated to be a destination for presenting media.

54. The medium of claim 52, wherein the step of establishing the communication link comprises establishing the communication link between a host device and the at least one client device.

55. The medium of claim 54, wherein the step of sending media data comprises sending a unicast stream of packets containing media data from the host device to the at least one client device, each packet having a timestamp and a sequence number.

56. The medium of claim 55, wherein the step of processing media data comprises queuing packets for processing by an engine at the at least one client device based on the sequence numbers.

57. The medium of claim 55, wherein the step of processing media data comprises:

determining the at least one client device whether a packet is missing based on the sequence numbers; and

sending a request for retransmission of a packet determined missing.

58. The medium of claim 57, wherein the step of sending media data further comprises sending a response to the request, the response containing the packet determined missing.

59. The medium of claim 55, wherein the step of sending media data comprises sending the packets containing media data at substantially regular intervals.

60. The medium of claim 52, wherein:

each of the devices comprises a clock, and

the step of control processing of media data comprises synchronizing each of the clocks with a reference clock.

61. The medium of claim 60, wherein the step of synchronizing comprises sending packets to each of the devices, the packets containing time information for correlating the local clocks with the reference clock.

62. The medium of claim 60, wherein the step of processing media data comprises the steps of:

constructing media data contained in received packets into a presentation timeline; and

processing media data in the presentation timeline based on timestamps in the packets correlated to the synchronized time of the local clock of the device.

63. The medium of claim 52, wherein the step of controlling processing of media data comprises adjusting a presentation time of processed media at each of the devices based on a maximum latency value determined for the devices.

64. The medium of claim 52, wherein the step of controlling processing of media data comprises:

determining a number of gaps in media occurring at each of the devices; and

disabling presentation of processed media or retransmission of lost packets of media data at a given device based on the number of gaps determined for that device.