MULTIMEDIA TIMELINE MODIFICATION IN NETWORKED CLIENT/SERVER SYSTEMS

• US 20020038374A1
• Filed: 09/15/1998
• Published: 03/28/2002
• Est. Priority Date: 09/15/1998
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: OMOIGUI, NOSAKHARE D., GUPTA, ANOOP
• Granted Patent: US 6,622,171 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355

Abstract

Multimedia content is streamed over a network system from a server computer to a client computer. The client allows a user to enter a variable playback speed and varies the speed at which the multimedia content is rendered at the client. Time-scale modification technology is used to maintain the original pitch of any audio content, thereby maintaining its intelligibility.

Claims

1. A method of obtaining and presenting multimedia content, comprising the following steps:

selecting multimedia content to be rendered at a network client;

accepting a speed designation at the network client from a human user independently of the selecting step;

streaming a composite media stream from a network server, the composite media stream representing the selected multimedia content;

rendering the composite media stream as it is streamed to produce the multimedia content at the network client;

varying the speed of the multimedia content depending on the speed designation from the human user.

2. A method as recited in claim 1, wherein:

the composite media stream has a timeline;

the step of varying the speed of the multimedia content is performed by altering the timeline of the composite media stream at the network server before streaming the composite media stream.

3. A method as recited in claim 1, wherein the step of varying the speed of the presented multimedia content is performed by receiving different composite media streams from the network server, wherein the different composite media streams have different timelines.

4. A method as recited in claim 1, comprising a further step of allowing the user to change the speed designation during rendering of the composite media stream.

5. A method as recited in claim 1, wherein the varying step uses time-scale-modification to change the speed of the multimedia content while maintaining its intelligibility.

6. A method as recited in claim 1, wherein:

the multimedia content includes audio content having an original pitch; and

the varying step uses time-scale-modification to change the speed of the multimedia content without significantly altering the original pitch of the audio content.

7. A method as recited in claim 1, further comprising:

storing multiple composite media streams at the network server corresponding to particular multimedia content, each of the multiple composite media streams having a timeline that is modified by a different degree;

selecting that composite media stream that most closely accords with the speed designation and streaming that composite media stream from the network server to the network client.

8. A method of obtaining and presenting multimedia content, comprising the following steps:

selecting a composite media stream from a network server to be rendered at a network client, the composite media stream having a plurality of individual media streams representing the multimedia content, at least one of the individual media streams comprising an audio stream representing audio content having an original pitch, the individual media streams having timelines that are synchronized with each other;

accepting a speed designation at the network client from a human user independently of the selecting step;

streaming the composite media stream from the network server to the network client;

modifying the timelines of the individual media streams in accordance with the speed designation while keeping the timelines synchronized and while maintaining the original pitch of the audio content;

rendering the composite media stream at the network client in accordance with the modified timelines of the individual media streams.

9. A method as recited in claim 8, wherein the step of modifying the timelines is performed by the network server before the step of streaming the composite media stream.

10. A method as recited in claim 8, wherein the step of modifying the timelines is performed by the network client after the step of streaming the composite media stream.

11. A method as recited in claim 8, wherein the step of accepting the speed designation is performing during the streaming step.

12. A method as recited in claim 8, wherein the streaming step comprises streaming the composite media stream at different rates depending on the speed designation from the human user.

14. A computer-readable storage medium as recited in claim 13, wherein the step of accepting the speed designation is performing during the streaming step.

15. A computer-readable storage medium as recited in claim 13, wherein the streaming step comprises streaming the composite media stream at different rates depending on the speed designation from the human user.

17. A networked client/server system as recited in claim 16, wherein each composite media stream includes at least an audio stream and a video stream.

19. A computer-readable storage medium as recited in claim 18, wherein the composite media stream includes at least an audio stream and a video stream.

20. A computer-readable storage medium as recited in claim 18, wherein the composing step includes a step of modifying the timeline of the composite media stream in accordance with the speed designation prior to the streaming step.

21. A computer-readable storage medium as recited in claim 18, wherein the composite media stream includes at least an audio stream representing audio content and wherein the composing step includes a step of modifying the timeline of the composite media stream while maintaining intelligibility of the audio content.

22. A computer-readable storage medium as recited in claim 18, further comprising:

storing multiple media streams at the network server corresponding to the multimedia content, said multiple media streams having timelines that are modified by different degrees and including streams corresponding to at least first and second media types, wherein media types of the first and second types can be rendered in combination to produce the multimedia content;

the step of composing a composite media stream comprising a step of selecting those stored media streams of the first and second types that have modified timelines most closely according with the speed designation.

23. A computer-readable storage medium as recited in claim 18, further comprising:

determining available bandwidth from the network server to the network client;

storing multiple media streams at the network server corresponding to the multimedia content, the multiple media streams including streams corresponding to at least first and second media types, wherein media types of the first and second types can be rendered in combination to produce the multimedia content;

the media streams of the first type having timelines that are modified by different degrees;

the media streams of the second type being of varying quality and requiring varying bandwidth;

wherein the composing step comprises;

selecting one of the media streams of the first type that most closely accords with the speed designation, wherein said selected one of the media streams of the first type consumes part of the available bandwidth;

selecting one of the media streams of the second type that requires no more bandwidth than the difference between the available bandwidth and the bandwidth consumed by the selected one of the media streams of the first type.

24. A computer-readable storage medium as recited in claim 18, further comprising:

determining available bandwidth from the network server to the network client;

storing a plurality of audio streams representing the multimedia content, the audio streams having timelines that are modified by different degrees;

storing a plurality of video streams representing the multimedia content, the video streams being of varying quality and requiring varying bandwidth;

wherein one of the audio streams and one of the video streams can be rendered in combination to produce the multimedia content;

wherein the composing step comprises;

selecting one of the audio streams having a timeline that most closely accords with the speed designation, wherein said selected audio stream consumes part of the available bandwidth;

selecting one of the video streams that requires no more bandwidth than the difference between the available bandwidth and the bandwidth consumed by the selected audio stream.

25. A computer-readable storage medium as recited in claim 18, further comprising:

determining available bandwidth from the network server to the network client;

storing an audio stream representing the multimedia content;

storing a plurality of video streams representing the multimedia content, the video streams being of varying quality and requiring varying bandwidth;

wherein the audio streams and one of the video streams can be rendered in combination to produce the multimedia content;

wherein the composing step comprises selecting one of the video streams that requires no more bandwidth than the difference between the available bandwidth and the bandwidth consumed by the audio stream when streamed at a rate that is proportional to the speed designation.

26. A computer-readable storage medium as recited in claim 18, further comprising:

determining available bandwidth from the network server to the network client;

storing an audio stream representing the multimedia content;

storing a plurality of video streams representing the multimedia content, the video streams having different timelines and requiring varying bandwidth;

wherein the audio streams and one of the video streams can be rendered in combination to produce the multimedia content;

wherein the composing step comprises selecting one of the video streams that requires no more bandwidth than the difference between the available bandwidth and the bandwidth consumed by the audio stream when streamed at a rate that is proportional to the speed designation.

28. A method as recited in claim 27, wherein the first individual media stream is an audio stream and the second individual media stream is a video stream.

29. A method as recited in claim 27, comprising a further step of modifying the timeline of the second individual media stream in accordance with the speed designation before it is streamed to the network client.

13. A computer-readable storage medium containing a program for obtaining and presenting multimedia content, the program having instructions that are executable by a network client to perform steps comprising:

selecting a composite media stream from a network server to be rendered at the network client, the composite media stream having a plurality of individual media streams representing the multimedia content, the individual media streams having timelines that are synchronized with each other;

accepting a speed designation from a human user independently of the selecting step;

streaming the composite media stream from the network server to the network client;

modifying the timelines of the individual media streams in accordance with the speed designation while keeping the timelines synchronized;

rendering the composite media stream at the network client in accordance with the modified timelines of the individual media streams.

16. A networked client/server system, comprising:

a network server;

a plurality of network clients that communicate with the network server over a data communications network;

a plurality of composite media streams available from the network server, each composite media stream comprising a plurality of individual media streams that can be rendered by the network clients to produce different types of user-perceivable media, the individual media streams including audio streams that can be rendered by the network clients to produce audio, such audio having an original pitch, the individual media streams having timelines that are synchronized with each other;

the network clients being responsive to user input to request identified composite media streams and to modify the timelines of the individual media streams while keeping the timelines synchronized with each other and while maintaining the original pitch of any audio produced from audio streams.

18. A computer-readable storage medium containing a program for streaming multimedia content to a network client, the program having instructions that are executable by a network server to perform steps comprising:

receiving a speed designation for playback of the multimedia content at a network client;

composing a composite media stream that represents the multimedia content, the composite media stream having a timeline that is modified in accordance with the speed designation;

streaming the timeline-modified composite media stream from the network server to the network client.

27. A method of obtaining and presenting multimedia content, comprising the following steps:

selecting multimedia content that is available from a network server, the multimedia content having first and second types of media content;

accepting a speed designation for playback of the multimedia content at a network client;

determining available bandwidth from the network server to the network client;

streaming a first individual media stream from the network server to the network client at a rate that is proportional to the speed designation, the first individual media stream representing the first type of media content and consuming part of the available bandwidth;

selecting a second individual media stream that represents the second type of media content, the second individual media stream being selected to have a quality that requires no more bandwidth than the difference between the available bandwidth and the bandwidth consumed by the first individual media stream;

streaming the second individual media stream from the network server to the network client;

modifying the timeline of the first individual media stream at the network client in accordance with the speed designation;

rendering the first and second individual media streams at the network client.

30. A method of streaming particular multimedia content from a network server to a network client, the method comprising the following steps:

storing multiple media streams at the network server corresponding to the multimedia content, said multiple media streams having timelines that are modified by different degrees and including streams corresponding to at least first and second media types, wherein a pair of the first and second types of media streams can be rendered to produce the multimedia content;

accepting a speed designation for playback of the multimedia content at a network client;

selecting those stored media streams of the first and second types that have modified timelines most closely according with the speed designation;

streaming the selected media streams from the network server to the network client.

31. A method as recited in claim 30, further comprising a step of further modifying the timelines of the selected media streams at the network client in accordance with the speed designation before rendering the selected media streams.