Production of a video stream with synchronized annotations over a computer network

• US 6,230,172 B1
• Filed: 09/03/1999
• Issued: 05/08/2001
• Est. Priority Date: 01/30/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Vellanki, Srinivas Prasad, Wang, Edward Yan-bing, Purnaveja, Audi, Gupta, Anoop, del Val, David, Chaddha, Navin
• Primary Examiner(s): Feild, Joseph H.
• Assistant Examiner(s): Kindred, Alford W.
• Application Number:
  US09/389,541
• Time in Patent Office: 613 Days
• Field of Search: 707/511-530, 345/302-340
• US Class Current: 715/205
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4886 for displaying a ticker, e.g. scrolling banner for news, stock exchange, weather data
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 5/44504 Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits

Abstract

The production of synchronization scripts and associated annotated multimedia streams for servers and client computers coupled to each other by a diverse computer network which includes local area networks (LANs) and/or wide area networks (WANs) such as the intermet. Annotated multimedia streams can include a compressed video stream for display in a video window, an accompanying compressed audio stream and annotations. Synchronization scripts include annotation streams for synchronizing the display of video streams with annotations, e.g., displayable events, such textual/graphical data in the form of HTML pages with Java applets to be displayed in one or more event windows. The producer includes a capture module and an author module for capturing video streams and generating annotation streams, respectively. The capture module compresses the video stream using a suitable compression format. Annotation streams include annotation frames which provide either pointer(s) to the event(s) of interest or include displayable data embedded within the annotation stream. Accordingly, each annotation frame includes either an event locator or an event data. In addition, each annotation frame includes an event time marker which corresponds to the time stamp(s) of associated video frame(s) within the video stream. Embedded displayable data include ticker tape data embedded within the annotation stream. Examples of event locators to displayable events include URL addresses pointing to HTML web pages. The video/audio streams and annotation streams are stored in stream server(s) for subsequent retrieval by client computer(s) in a coordinated manner, so that the client computer(s) is able to synchronously display the video frames and displayable event(s) in a video window and event window(s), respectively. In one implementation, annotation streams include a flipper stream for locating HTML pages and a ticker stream which include ticker (tape) data.

Claims

1. A computer-readable medium useful in association with a client computer having a display device, the computer-readable medium having computer-executable instructions which cause the computer system to perform a method comprising:

receiving a video stream from a stream server coupled to the client computer via a computer network, said video stream including a plurality of video frames, each said video frame including a time stamp;

sequentially displaying said video frames on said display device, starting from an initial video frame of said video stream;

selecting a content label from a table of contents displayed on said display device, said content label providing an index into an intermediate video frame of said video stream, said index based on the time stamp of said intermediate video frame;

communicating said index to said stream server;

receiving said video stream from said stream server, starting from said intermediate video frame; and

sequentially displaying said video frames on said display device, starting with said intermediate video frame.

9. The computer readable medium of claim 1, wherein said intermediate video frame includes an I-frame.

10. The computer readable medium of claim 1, wherein said intermediate video frame includes a P-frame.

2. A computer-readable medium useful in association with a stream server, said stream server coupled to a client computer via a computer network, said client computer including a display device, the computer-readable medium having computer-executable instructions which cause the stream server to perform a method comprising:

streaming said video stream to said client computer for sequential display on said display device, starting from an initial video frame of a plurality of video frames of said video stream, each said video frame including a time stamp;

receiving an index from said client computer, said index associated with a selectable content label from a table of contents displayed on said display device, said index based on the time stamp of an intermediate video frame of said video stream; and

streaming said video stream to said client computer for sequential display on said display device, starting from said intermediate video frame.

11. The computer-readable medium of claim 2, wherein said intermediate video frame includes an I-frame.

12. The computer readable medium of claim 2, wherein said intermediate video frame includes a P-frame.

3. A method for playing a stream of data having a number of frames, comprising:

receiving the stream of data from a server;

sequentially playing the frames, starting from an initial frame of the stream of data;

selecting a content label from a table of contents, the table of content being displayed on a display device of a computer, the content label providing an index into an intermediate frame of the stream of data, the index based on a time stamp of the intermediate frame;

communicating the index to the server;

receiving the stream of data from the server, starting from the intermediate frame; and

sequentially playing the frames, starting with the intermediate frame.

13. The method of claim 3, wherein said intermediate video frame includes an I-frame.

14. The method of claim 3, wherein said intermediate video frame includes a P-frame.

4. A method of streaming a stream of data having a number of frames, comprising:

streaming the stream of data to a computer for sequential playing, starting from an initial frame of the stream of data;

receiving an index from the computer, the index associated with a selectable content label from a table of contents, the table of contents being displayed on a display device of the computer, the index based on a time stamp of an intermediate frame of the stream of data; and

streaming the stream of data to the computer for sequential playing, starting, from the intermediate frame.

5. A computer comprising:

a playout buffer configured to receive a stream of data from a server, the stream of data including a number of frames, such that the computer plays the frames, starting from an initial frame of the stream of data;

a display device configured to display a table of contents, the table of contents including a selectable content label, the selectable content label providing an index into an intermediate frame of the stream of data, the index based on a time stamp of the intermediate frame; and

a transmitter configured to communicate the index to the server, thereby causing the server to begin transmitting the stream of data to the computer, starting from the intermediate frame and causing the computer to play the frames of the stream of data, starting with the intermediate frame.

6. A server comprising:

a streamer configured to store and stream a stream of data to a computer, the stream of data having a number of frames, the stream of data enabling the computer to sequentially play the frames on the computer, starting from an initial frame of the stream of data; and

a receiver configured to receive an index from the computer, the index associated with a selectable content label from a table of contents displayed on a display device of the computer, the index based on a time stamp of an intermediate frame of the stream of data,

wherein the index causes the streamer to begin streaming the stream of data to the computer and enabling the computer to sequentially play the frames, starting from the intermediate frame.

7. A computer-readable medium having computer-executable instructions comprising:

receiving a stream of data from a server;

sequentially playing the frames, starting from an initial frame of the stream of data;

selecting a content label from a table of contents, the table of content being displayed on a display device of a computer, the content label providing an index into an intermediate frame of the stream of data, the index based on a time stamp of the intermediate frame;

communicating the index to the server;

receiving the stream of data from the server, starting from the intermediate frame; and

sequentially playing the frames, starting with the intermediate frame.

8. A computer-readable medium having computer-executable instructions comprising:

streaming a stream of data to a computer for sequential playing, starting from an initial frame of the stream of data;

receiving an index from the computer, the index associated with a selectable content label from a table of contents, the table of contents being displayed on a display device of the computer, the index based on a time stamp of an intermediate frame of the stream of data; and

streaming the stream of data to the computer for sequential playing, starting from the intermediate frame.

15. A computer-readable medium useful in association with a client computer having a display device, the computer-readable medium comprising:

receiving a video stream from a stream server coupled to the client computer via a computer network, said video stream including a plurality of video frames, each said video frame including a time stamp;

sequentially displaying said video frames on said display device, starting from an initial video frame of said video stream;

selecting a content label from a table of contents displayed on said display device;

locating an I-frame in the video stream that is closest to the content label, wherein the I-frame has an associated timestamp;

communicating said associated timestamp to said stream server;

receiving said video stream from said stream server, starting from said intermediate video frame; and

sequentially displaying said video frames on said display device, starting with said intermediate video frame.

16. A client computer comprising:

a playout buffer configured to receive a stream of data from a server, the stream of data including a number of frames, such that the computer plays the frames, starting from an initial frame of the stream of data;

a display device configured to display a table of contents, the table of contents including a selectable content label; and

a transmitter configured to communicate a timestamp of a closest frame to the server, wherein the closest frame is a closest P-frame to the selectable content label, thereby causing the server to begin transmitting the stream of data to the computer, starting from the closest frame and causing the client computer to play the frames of the stream of data, starting with the closest frame.

17. The client computer of claim 16, wherein the stream of data includes a video stream.

18. The client computer of claim 17, wherein the stream of data further includes an annotation stream associated with the video stream.