System and method for synchronizing enhancing content with a video program using closed captioning

• US 6,637,032 B1
• Filed: 01/06/1997
• Issued: 10/21/2003
• Est. Priority Date: 01/06/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Feinleib, David
• Primary Examiner(s): Harvey, David E.
• Application Number:
  US08/779,270
• Time in Patent Office: 2,479 Days
• Field of Search: H04/N.5445, 348/13, 348/12, 348/460, 348/461, 348/468, 348/563, 348/564, 348/565, 348/552, 345/327, 345/328, 725/109, 725/110, 725/112, 725/113, 725/111
• US Class Current: 725/110
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles
• CPC Class Codes: H04N 7/0887 for the transmission of programme or channel identifying signals
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

A system and method uses the closed captioning script to synchronize supplemental data with specified junctures in a video program. A parser parses the closed captioning script to identify a set of unique phrases, with each phrase having the same number of words. A program producer decides what points in the video program to introduce enhancing content. The producer associates supplemental data used to activate the enhancing content with specific key phrases of the closed captioning script that correspond to the desired points in the program. The parser creates a key phrase data file which contains a listing of the key phrases and their association to the supplemental data. The key phrase data file is delivered to viewer computing units at users'"'"' homes. When the program is played, the viewer computing unit monitors the closed captioning script to detect the key phrases listed in the key phrase data file. Upon detection of a particular key phrase, the viewer computing unit accesses the key phrase data file to retrieve the supplemental data associated with the particular key phrase. The viewer computing unit executes an enhancement action according to the supplemental data to synchronize the enhancement action with the video program. According to one implementation, the key phrase detector multicasts the enhancement action to a multicast address. A listener in the form of an ActiveX™ control listens to the multicast address to receive and handle any enhancement action supported by the supplemental data. The listener can be embedded in a container, such as an HTML page or other hypermedia document.

Claims

1. A method for synchronizing enhancing content with primary content having a closed captioning script, comprising the following steps:

relating supplemental data used to activate the enhancing content to key phrases of the closed captioning script;

detecting the key phrases from the closed captioning script during presentation of the primary content; and

activating the enhancing content according to the supplemental data related to the detected key phrases without user interaction to synchronize the enhancing content with the primary content.

2. A method as recited in claim 1, wherein the relating step comprises associating the supplemental data with the key phrases in a data file.

3. A method as recited in claim 1, wherein the detecting step comprises examining sequential groupings of words in the closed captioning script to identify the key phrases.

5. A method as recited in claim 1, wherein the supplemental data comprises executable code, and the activating step comprises the step of launching the executable code.

6. A method as recited in claim 1, wherein the activating step comprises the step of displaying the supplemental data concurrently with the primary content.

8. A method as recited in claim 1, wherein the key phrases comprise sequences of words in the closed captioning script.

9. A method as recited in claim 1, wherein the key phrases comprise character strings in the closed captioning script.

10. Computers programmed to perform the steps of the method as recited in claim 1.

11. Computer-readable media having computer-executable instructions for performing the steps of the method as recited in claim 1.

4. A method for synchronizing enhancing content with primary content having a closed captioning script, comprising the following steps:

relating a hyperlink for a target resource containing enhancing content to key phrases of the closed captioning script;

detecting the key phrases from the closed captioning script during presentation of the primary content; and

activating the hyperlink related to the detected key phrases without user interaction to synchronize the enhancing content with the primary content.

7. A method as recited in claim 4, further comprising the following steps:

presenting the primary content within a hypermedia document; and

controlling placement of the primary content within the hypermedia document using the supplemental data.

12. A method for producing an enhanced program in which supplemental data is made available to enhance primary content, the primary content containing a closed captioning script, comprising the following steps:

determining, during production of an enhanced program, a key phrase in the closed captioning script; and

associating the supplemental data to the key phrase in the closed captioning script during the production.

13. A method as recited in claim 12, wherein the determining step comprises deriving multiple key phrases which each have the same phrase length.

14. A method as recited in claim 12, wherein the determining step comprises compiling a set of unique key phrases from the closed captioning script, each key phrase having the same number of words.

15. A method as recited in claim 12, further comprising the following steps:

marking spots in the closed captioning script for presentation of enhancing content; and

the determining step comprises determining the key phrases at the spots in the closed captioning script.

16. A method as recited in claim 12, further comprising the step of generating a data file containing the supplemental data associated with the key phrase.

17. A storage medium storing the data file constructed according to the steps in the method as recited in claim 16.

18. A computer programmed to perform the steps of the method as recited in claim 12.

19. A computer-readable media having computer-executable instructions for performing the steps of the method as recited in claim 12.

20. A method for enhancing primary content having a closed captioning script, comprising the following steps:

detecting a key phrase in the closed captioning script, the key phrase being related to enhancing content; and

enhancing the primary content with the enhancing content related to the detected key phrase without user interaction to synchronize the enhancing content to the primary content.

21. A method as recited in claim 20, wherein the detecting step comprises:

examining sequential groupings of words in the closed captioning script to identify the key phrases; and

comparing the groupings of words with a preconstructed list of key phrases to identify the key phrases in the closed captioning script.

23. A method as recited in claim 20, wherein the enhancing step comprises the step of launching executable code.

24. A method as recited in claim 20, wherein the enhancing step comprises the step of concurrently displaying the primary and enhancing content.

26. A computer programmed to perform the steps of the method as recited in claim 20.

27. A computer-readable media having computer-executable instructions for performing the steps of the method as recited in claim 20.

22. A method for enhancing primary content having a closed captioning script, comprising the following steps:

detecting a key phrase in the closed captioning script, the key phrase being related to enhancing content; and

activating a hyperlink to a target resource related to the detected key phrase without user interaction to synchronize enhancing content from the target resource with the primary content.

25. A method as recited in claim 22, further comprising the following steps:

presenting the primary content within a hypermedia document; and

controlling placement of the primary content within the hypermedia document using the enhancing content.

28. A method for enhancing a video program having a closed captioning script, comprising the following steps:

determining a key phrase in the closed captioning script;

associating supplemental data to the key phrase in the closed captioning script;

creating a key phrase data file which contains the key phrase and the associated supplemental data;

delivering the key phrase data file to a viewer computing unit;

playing the video program at the viewer computing unit;

detecting the key phrase in the closed captioning script;

obtaining the supplemental data associated with the key phrase from the key phrase data file; and

executing an enhancement action without user interaction according to the obtained supplemental data to synchronize the enhancement action with the video program.

30. A method as recited in claim 28, wherein the supplemental data comprises executable code, and the executing step comprises the step of launching the executable code.

33. A method as recited in claim 28, wherein the executing step comprises the following steps:

multicasting the enhancement action to a multicast address;

listening to the multicast address to receive the enhancement action; and

performing the enhancement action.

34. Computers programmed to perform the steps of the method as recited in claim 28.

35. Computer-readable media having computer-executable instructions for performing the steps of the method as recited in claim 28.

36. A storage medium storing the key phrase data file constructed according to the steps in the method as recited in claim 28.

29. A method for enhancing a video program having a closed captioning script, comprising the following steps:

determining a key phrase in the closed captioning script;

associating a hyperlink to a target resource to the key phrase in the closed captioning script;

creating a key phrase data file which contains the key phrase and the associated hyperlink;

delivering the key phrase data file to a viewer computing unit;

playing the video program at the viewer computing unit;

detecting the key phrase in the closed captioning script;

obtaining the hyperlink associated with the key phrase from the key phrase data file; and

activating the hyperlink to the target resource to synchronize an enhancement action with the video program.

31. A method for enhancing a video program having a closed captioning script, comprising the following steps:

determining a key phrase in the closed captioning script;

associating supplemental data to the key phrase in the closed captioning script;

creating a key phrase data file which contains the key phrase and the associated supplemental data;

delivering the key phrase data file to a viewer computing unit;

playing the video program at the viewer computing unit;

detecting the key phrase in the closed captioning script;

obtaining the supplemental data associated with the key phrase from the key phrase data file; and

displaying the supplemental data in synchronization and concurrently with the video program.

32. A method for enhancing a video program having a closed captioning script, comprising the following steps:

determining a key phrase in the closed captioning script;

associating supplemental data to the key phrase in the closed captioning script;

creating a key phrase data file which contains the key phrase and the associated supplemental data;

delivering the key phrase data file to a viewer computing unit;

playing the video program at the viewer computing unit;

presenting the video program within a hypermedia document;

detecting the key phrase in the closed captioning script;

obtaining the supplemental data associated with the key phrase from the key phrase data file; and

controlling placement of the video program within the hypermedia document using the supplemental data.

37. A system for synchronizing enhancing content with primary content having a closed captioning script, comprising:

means for relating supplemental data to key phrases of the closed captioning script, the supplemental data being used to activate the enhancing content;

means for detecting the key phrases from the closed captioning script during presentation of the primary content; and

means for automatically activating the enhancing content according to the supplemental data related to the detected key phrases without user interaction to introduce the enhancing content at times approximately concurrent with occurrence of the key phrases in the primary content.

38. A system for producing an enhanced program in which supplemental data is made available to enhance a video program, the video program containing a closed captioning script, comprising:

a key phrase identifier to identify one or more key phrases from the closed captioning script;

a key phrase linker to associate the supplemental data to the one or more key phrases identified from the closed captioning script by the key phrase identifier;

wherein the key phrase linker multicasts the supplemental data to a multicast address; and

a multicast listener to receive the supplemental data sent to the multicast address.

39. A system as recited in claim 38, wherein the key phrase identifier comprises a parser.

40. A system as recited in claim 38, wherein the key phrase identifier comprises a parser configured to produce a set of unique phrases having the same phrase length.

41. A system as recited in claim 38, wherein the key phrase linker creates a data file having the key phrases associated with the supplemental data.

42. A system as recited in claim 38, further comprising a processor and a data storage medium, wherein the key phrase identifier, the key phrase linker, and the multicast listener comprise computer-executable instructions stored in the data storage medium and executable on the processor.

43. For use with a viewer computing unit having a data storage medium and a processor, a program enhancement system for enhancing a video program having a closed captioning script, the program enhancement system comprising:

an association table stored in the storage medium, the association table containing key phrases in the closed captioning script listed in association with supplemental data used to enhance the video program; and

a key phrase detector executable on the processor to monitor the closed captioning script as the video program is playing and to detect the key phrases listed in the association table, whereupon detection of a particular key phrase and without user interaction, the association table can be accessed to retrieve the supplemental data associated with the particular key phrase to synchronize utilization of the supplemental data with a specific spot in the video program corresponding to the particular key phrase.

44. A program enhancement subsystem as recited in claim 43, wherein the key phrase detector comprises a parser.

45. A program enhancement subsystem as recited in claim 43, wherein the key phrase detector examines successive groups of sequential words in the closed captioning script and compares each group to the key phrases in the association table to detect occurrence of the key phrases in the closed captioning script.

46. For use with a viewer computing unit having a data storage medium and a processor, a program enhancement system for enhancing a video program having a closed captioning script, the program enhancement system comprising:

an association table stored in the storage medium, the association table containing key phrases in the closed captioning script listed in association with supplemental data used to enhance the video program, wherein the supplemental data comprises data selected from a group comprising a hyperlink to a target resource, text data, graphical data, and executable code; and

a key phrase detector executable on the processor to monitor the closed captioning script as the video program is playing and to detect the key phrases listed in the association table, whereupon detection of a particular key phrase, the association table can be accessed to retrieve the supplemental data associated with the particular key phrase to synchronize utilization of the supplemental data with a specific spot in the video program corresponding to the particular key phrase.

47. For use with a viewer computing unit having a data storage medium and a processor, a program enhancement system for enhancing a video program having a closed captioning script, the program enhancement system comprising:

an association table stored in the storage medium, the association table containing key phrases in the closed captioning script listed in association with supplemental data used to enhance the video program;

a key phrase detector executable on the processor to monitor the closed captioning script as the video program is playing and to detect the key phrases listed in the association table, whereupon detection of a particular key phrase, the association table can be accessed to retrieve the supplemental data associated with the particular key phrase to synchronize utilization of the supplemental data with a specific spot in the video program corresponding to the particular key phrase, the key phrase detector multicasting the supplemental data to a multicast address; and

a multicast listener executable on the processor to receive the supplemental data sent to the multicast address.

48. For use with a viewer computing unit having a processor, a program enhancement subsystem for enhancing a video program having a closed captioning script, the program enhancement subsystem comprising:

a detector executable on the processor to monitor the closed captioning script as the video program is playing and to detect supplemental data embedded in the closed captioning script, whereupon detection and without user interaction, the processor executes an enhancement action based on the supplemental data to synchronize the enhancement action with a specific spot in the video program corresponding to the placement of the supplemental data within the closed captioning script.

49. A computer-readable memory having a key phrase data structure, the key phrase data structure comprising:

a key phrase data field containing key phrases abstracted from a closed captioning script of a video program;

a supplemental data field containing supplemental data for enhancing a video program; and

the key phrase data structure being organized so that the supplemental data in the supplemental data field is associated with corresponding key phrases in the key phrase data field.

50. A computer-readable memory as recited in claim 49, wherein each key phrase has the same number of words, the key phrase data structure further comprising a minimum phrase length data field containing a length of each key phrase.