Resynchronizing globally unsynchronized multimedia streams

• US 6,985,966 B1
• Filed: 03/29/2000
• Issued: 01/10/2006
• Est. Priority Date: 03/29/2000
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gupta, Anoop, Omoigui, Nosakhare D.
• Primary Examiner(s): Etienne, Ario
• Assistant Examiner(s): Todd, Gregory G.
• Application Number:
  US09/538,611
• Time in Patent Office: 2,113 Days
• Field of Search: 709231-234, 709/217, 709/248, 709/204, 345/725, 345/704, 345/748, 345/753, 345/756, 345/730
• US Class Current: 709/248
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4398 involving reformatting operations of audio signals details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44029 for generating different versions
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896

Abstract

Media streams are provided from a server computer to client computers in a network system. The media streams are monitored and the streams becoming globally unsynchronized is detected. When the streams become unsynchronized, action is taken to resynchronize the streams, such as by altering the presentation timeline of at least one of the media streams.

Claims

1. A method for use in a client computer, the method comprising:

receiving at a first client computer a media stream from a server computer in a network system, the media stream being simultaneously streamed to multiple client computers including the first client computer and a second client computer;

buffering at the first client computer media of the received media stream;

detecting at the first client computer when the media stream received from the server computer in the network system has become globally unsynchronized with the media stream being simultaneously streamed to the second client computer, the media stream received by the first client computer being globally unsynchronized with the media stream being streamed to the second client computer when the media in the buffer at the first client computer is greater than a first amount or is less than a second amount, and wherein the first client computer determines that it is globally unsynchronized independently of the media stream being streamed to the second client computer; and

altering at the first client computer a presentation timeline of the media stream in order to resynchronize the media stream of the first client computer with the media stream of the second client computer.

2. A method as recited in claim 1, wherein the altering includes altering the media stream.

3. A method as recited in claim 1, wherein the altering comprises compressing a presentation timeline of the media stream.

4. A method as recited in claim 1, wherein the altering comprises increasing the speed at which the media stream is rendered.

5. A method as recited in claim 1, wherein the altering comprises omitting selected frames from the media stream.

6. A method as recited in claim 1, wherein the altering comprises using time-scale-modification to remove data from or add data to the media stream.

7. A method as recited in claim 1, wherein the altering comprises jumping ahead to a later presentation time.

8. A method as recited in claim 1, wherein the altering comprises pausing the presentation of the media stream.

9. A method as recited in claim 1, further comprising:

storing at least a portion of the media stream in a data buffer; and

wherein the detecting comprises comparing the amount of data stored in the data buffer with a threshold.

10. A method as recited in claim 1, further comprising:

detecting when the media stream has been globally resynchronized; and

altering the presentation of the media stream when the media stream has been globally resynchronized.

11. A method as recited in claim 10, wherein the altering of the presentation timeline of the media stream when the media stream has been globally resynchronized comprises altering the presentation timeline to be the same as it was when the media stream was globally unsynchronized.

12. A computer-readable memory which directs the client computer to perform the steps of the method as recited in claim 1.

13. An apparatus for use in a network system, the apparatus comprising:

a receiving component to receive at a first client computer a plurality of media streams from a server computer in the network system, the media streams being simultaneously streamed to multiple client computers including the first client computer and a second client computer;

buffers at the first client computer for storing media of the received media streams;

a synchronizing component, coupled to the buffers, to determine if the plurality of media streams at the first client computer have become globally unsynchronized with the media streams being simultaneously streamed to the second client computer, a media stream being globally unsynchronized when the media in a buffer at the first client computer is greater than a first amount or less than an second amount, and wherein the first client computer determines that it is globally unsynchronized independently of the media stream being streamed to the second client computer; and

a timeline modification component, coupled to the synchronizing component, to alter the presentation timeline of at least one of the media stream of the first client computer if the media streams of the first client computer becomes globally unsynchronized with the media stream of the second client computer.

14. An apparatus as recited in claim 13, wherein the timeline modification component is to alter the timeline of the at least one media stream by compressing the timeline.

15. An apparatus as recited in claim 13, wherein the timeline modification component is to alter the timeline of the at least one media stream by omitting selected frames from the media stream.

16. An apparatus as recited in claim 13, wherein the timeline modification component is to alter the timeline of the at least one media stream by using time-scale-modification to remove data from or add data to the media stream.

17. A computer-readable storage medium containing a program for resynchronizing a media stream, the program having instructions that are executable by a network client to perform steps comprising:

receiving, from a server computer in the network, at a first network client, a composite media stream including a plurality of media streams, the plurality of media streams being simultaneously streamed to multiple network clients including the first network client and a second network client;

buffering, at the first network client, a media stream;

detecting, at the first network client, when the media streams being streamed to the first network client has become globally unsynchronized with the media stream being simultaneously streamed to the second network client, wherein the media stream received by the first network client is globally unsynchronized with the media stream received by the second network client when the media in the buffer at the first network client is greater than a first amount or is less than a second amount, and wherein the first network client determines that it is globally unsynchronized independently of the media stream being streamed to the second network client; and

altering, at the network client, a timeline of the media stream of the first network client in order to resynchronize the media streams of the first network client and the second network client.

18. A computer-readable storage medium as recited in claim 17, wherein the detecting comprises comparing current presentation times of the plurality of media streams to one another, and wherein the altering comprises altering the media stream of the plurality of media streams having a presentation time that is lagging behind the presentation times of one or more of the other media streams.

19. A computer-readable storage medium as recited in claim 17, wherein the detecting comprises comparing current presentation times of the plurality of media streams to one another, and wherein the altering comprises altering the media stream of the plurality of media streams having a presentation time that is ahead of the presentation times of one or more of the other media streams.

20. A computer-readable storage medium as recited in claim 17, wherein the altering comprises altering each media stream of the plurality of media streams.

21. A computer-readable storage medium as recited in claim 17, wherein the altering comprises compressing the timeline of the at least one media stream.

22. A computer-readable storage medium as recited in claim 17, wherein the altering comprises increasing the speed at which the at least one media stream is rendered.

23. A computer-readable storage medium as recited in claim 17, wherein the altering comprises omitting selected frames from the at least one media stream.

24. A computer-readable storage medium as recited in claim 17, wherein the altering comprises using time-scale-modification to remove data from the media stream.

25. A computer-readable storage medium as recited in claim 17, the program having instructions that are executable by the network client to further perform steps comprising:

detecting when the media streams have been resynchronized; and

altering the timeline of the at least one media stream when the media streams have been resynchronized.

26. A computer-readable storage medium as recited in claim 25, wherein the step of altering the timeline of the at least one media stream when the media streams have been resynchronized comprises altering the timeline to be the same as it was when the at least one media stream was globally unsynchronized.