INTERLEAVED MULTIPLE MULTIMEDIA STREAM FOR SYNCHRONIZED TRANSMISSION OVER A COMPUTER NETWORK

• US 20010013068A1
• Filed: 03/25/1997
• Published: 08/09/2001
• Est. Priority Date: 03/25/1997
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: CHOU, PHILIP A., KLEMETS, ANDERS EDGAR
• Granted Patent: US 6,449,653 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

The production of an interleaved multimedia stream for servers and client computers coupled to each other by a diverse computer network which includes local area networks (LANs) and/or wide area networks (WANs) such as the internet. Interleaved multimedia streams can include compressed video frames for display in a video window, accompanying compressed audio frames and annotation frames. In one embodiment, a producer captures separate video/audio frames and generates an interleaved multimedia file. In another embodiment, the interleaved file include annotation frames which provide either pointer(s) to the event(s) of interest or include displayable data embedded within the annotation stream. The interleaved file is then stored in the web server for subsequent retrieval by client computer(s) in a coordinated manner, so that the client computer(s) is able to synchronously display the video frames and displayable event(s) in a video window and event window(s), respectively. In some embodiments, the interleaved file includes packets with variable length fields, each of which are at least one numerical unit in length.

Claims

1. In a computer having a processor and memory, said computer useful in association with a web server coupled to a client computer via a network, a method for producing an interleaved multimedia file from a video file and an audio file, the method comprising the steps of:

if a video frame buffer is empty, then retrieving a first video frame from the video file, said first video frame including a video timestamp;

if an audio frame buffer is empty, then retrieving a first audio frame from the audio file, said first audio frame including an audio timestamp; and

if the video timestamp is less than or equal to the audio timestamp, then

writing the first video frame to a first packet of the interleaved file; and

retrieving a second video frame from the video file;

else if the audio timestamp is less than or equal to the video timestamp, then

writing the first audio frame to a second packet of the interleaved file; and

retrieving a second audio frame from the audio file.

2. The method of

claim 1

wherein said first and second packet has a variable packet length field.

3. The method of

claim 2

wherein the size of the variable packet length field is at least one numerical unit, and said step of writing the first video frame to the first packet includes the steps of:

if the size of the first video frame is less than one numerical unit, then

writing the size into a length field of the variable packet length field of the first packet;

if the size of the first video frame is between one numerical unit and two numerical units, then

writing a null number into a null field of the variable packet length field of the first packet; and

writing the size into the length field of the variable packet length field of the first packet; and

if the size of the first video frame is greater than two numerical units, then

writing three null numbers into the null field of the variable packet length field of the first packet; and

writing the size into the length field of the variable packet length field of the first packet.

4. The method of

claim 1

further comprising the step of writing an annotation frame from an annotation file into said interleaved file.

5. A producer useful for generating an interleaved file configured to provide a synchronized playback of a video file and an audio file on a client computer, the producer comprising a capture module configured to generate an interleaved file from video frames from the video file and audio frames from the audio file, based on timestamps of the video frames and the audio frames.

6. The producer of

claim 5

further comprising an author module configured to combined said interleaved file with a plurality of annotation frames, based on the timestamps of the video frames, the audio frames and the annotation frames.

7. The producer of

claim 5

wherein said video and audio frames are stored in packets with variable packet length fields.

8. A computer-readable medium useful in association with a computer system having a processor and memory, the computer-readable medium comprising computer-readable code instructions configured to cause said computer system to execute the steps of:

if a video frame buffer is empty, then retrieving a first video frame from a video file, said first video frame including a video timestamp;

if an audio frame buffer is empty, then retrieving a first audio frame from an audio file, said first audio frame including an audio timestamp; and

if the video timestamp is less than or equal to the audio timestamp, then

writing the first video frame to a first packet of an interleaved file; and

retrieving a second video frame from the video file;

else if the audio timestamp is less than or equal to the video timestamp, then

writing the first audio frame to a second packet of the interleaved file; and

retrieving a second audio frame from the audio file.

9. The computer-readable medium of

claim 8

wherein said first and second packet has a variable packet length field.

10. The computer-readable medium of

claim 9

wherein the size of the variable packet length field is at least one numerical unit, and said step of writing the first video frame to the first packet includes the steps of:

if the size of the first video frame is less than one numerical unit, then

writing the size into a length field of the variable packet length field of the first packet;

if the size of the first video frame is between one numerical unit and two numerical units, then

writing a null number into a null field of the variable packet length field of the first packet; and

writing the size into the length field of the variable packet length field of the first packet; and

if the size of the first video frame is greater than two numerical units, then

writing three null numbers into the null field of the variable packet length field of the first packet; and

writing the size into the length field of the variable packet length field of the first packet.

11. The computer readable medium of

claim 8

further comprising computer-readable code instructions configured to cause said computer system to execute the step of writing an annotation frame from an annotation file into said interleaved file.