Methods and apparatus for keyword-based, non-linear navigation of video streams and other content

• US 9,407,892 B2
• Filed: 09/12/2011
• Issued: 08/02/2016
• Est. Priority Date: 09/12/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Mo, Stanley, Ochoa, Claudio J., Szilagyi, Victor, Smith, Elliot
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Tekle, Daniel
• Application Number:
  US13/994,838
• Publication Number:
  US 20130272676A1
• Time in Patent Office: 1,786 Days
• US Class Current: 1/1
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/432 Content retrieval operation from a local storage medium, e.g. hard-disk details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8405 represented by keywords
• CPC Class Codes: H04N 9/79 Processing of colour television signals in connection with recording
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

In one example embodiment, a program navigation system involves a television (TV) set (or other audio/visual presentation device) and a mobile device. While the TV set is playing an audio/visual program, a keyword recognition module may extract keywords from textual data associated with the program. The textual data may include textual words that match words spoken in the program. Time markers for the keywords may also be saved. A time marker may indicate a play time within the program for the corresponding keyword. The TV set may automatically transmit the keywords to the mobile device, and the mobile device may automatically display the keywords in a user interface. A user may then select one of the keywords. In response, the mobile device may automatically cause the TV set to jump to a portion of the program containing the selected keyword. Other embodiments are described and claimed.

Claims

1. A method for audio and visual program navigation, the method comprising:

at a mobile computing device, wirelessly receiving, from an audio and visual presentation system, a stream of keyword data for an audio and visual program currently being played by the audio and visual presentation system, wherein the stream of keyword data comprises textual keywords that match spoken words from audio content of the audio and visual program;

at the mobile computing device, automatically displaying two or more of the keywords from the audio and visual presentation system in a user interface;

at the mobile computing device, receiving user input indicating that a user has selected one of the displayed keywords; and

in response to receiving the user input at the mobile computing device, automatically causing the audio and visual presentation system to jump to a portion of the audio and visual program that contains the selected keyword.

2. A method according to claim 1, wherein the keyword data comprises keywords extracted from closed caption data for the audio and visual program by the audio and visual presentation system.

3. A method according to claim 1, wherein the audio and visual presentation system comprises a display device in communication with at least one decoding device from the group consisting of:

a television tuner;

an optical disc player; and

a digital video recorder.

4. A method according to claim 1, wherein:

the keyword data further comprises time markers associated with corresponding keywords; and

the operation of automatically causing the audio and visual presentation system to jump to the portion of the audio and visual program that contains the selected keyword comprises;

generating a program control command instructing the audio and visual presentation system to jump to the selected portion of the audio and visual program, based on the selected keyword and the corresponding time marker; and

sending the program control command to the audio and visual presentation system via a wireless communications interface.

5. A method according to claim 1, further comprising:

establishing a wireless communication channel between the mobile computing device and the audio and visual presentation system;

using the wireless communication channel to receive the keyword data for the audio and visual program from the audio and visual presentation system; and

using the wireless communication channel to cause the audio and visual presentation system to jump to the portion of the audio and visual program that contains the selected keyword.

6. A method according to claim 1, wherein:

the operation of automatically displaying two or more of the keywords in the user interface comprises scrolling new keywords onto the user interface and scrolling old keywords off of the user interface as the mobile computing device receives new keywords from the audio and visual presentation system; and

in response to user input, scrolling old keywords back onto the user interface, to allow the user to utilize one of the old keywords to select a corresponding portion of the audio and visual program to be played by the audio and visual presentation system.

7. A method according to claim 1, further comprising:

after receiving at least one keyword for the audio and visual program from the audio and visual presentation system;

at the mobile computing device, automatically retrieving, from a public wide area network, information relating to the at least one keyword; and

at the mobile computing device, automatically displaying at least some of the information from the public wide area network relating to the at least one keyword.

8. A method according to claim 7, wherein:

the operation of automatically retrieving information relating to the at least one keyword comprises;

automatically retrieving reference information about a meaning of the at least one keyword from an online encyclopedia; and

automatically retrieving purchasing information about products pertaining to the at least one keyword from a website of a vendor who offers said products for sale over the public wide area network; and

the operation of automatically displaying at least some of the information from the public wide area network relating to at least one of the keywords comprises simultaneously displaying at least some of the reference information and at least some of the purchasing information at the mobile computing device.

9. A method according to claim 8, further comprising:

receiving user input requesting deactivation of a particular source of information on the public wide area network; and

automatically retrieving information only from sources of information on the public wide area network for which deactivation has not been requested.

10. An article of manufacture comprising:

a tangible machine accessible medium; and

instructions in the machine accessible medium which, when executed by a mobile device, enable the mobile device to perform operations comprising;

wirelessly receiving, from an audio and visual presentation system, a stream of keyword data for an audio and visual program currently being played by the audio and visual presentation system, wherein the stream of keyword data comprises textual keywords that match spoken words from audio content of the audio and visual program;

automatically displaying two or more of the keywords from the audio and visual presentation system in a user interface;

receiving user input indicating that a user has selected one of the displayed keywords; and

in response to receiving the user input, automatically causing the audio and visual presentation system to jump to a portion of the audio and visual program that contains the selected keyword.

11. An article of manufacture according to claim 10, wherein:

the keyword data further comprises time markers associated with corresponding keywords; and

the operation of automatically causing the audio and visual presentation system to jump to the portion of the audio and visual program that contains the selected keyword comprises;

generating a program control command instructing the audio and visual presentation system to jump to the selected portion of the audio and visual program, based on the selected keyword and the corresponding time marker; and

sending the program control command to the audio and visual presentation system via a wireless communications interface.

12. An article of manufacture according to claim 10, wherein the instructions enable the mobile device to perform further operations comprising:

establishing a wireless communication channel between the mobile computing device and the audio and visual presentation system;

using the wireless communication channel to receive the keyword data for the audio and visual program from the audio and visual presentation system; and

using the wireless communication channel to cause the audio and visual presentation system to jump to the portion of the audio and visual program that contains the selected keyword.

13. An article of manufacture according to claim 10, wherein:

the operation of automatically displaying two or more of the keywords in the user interface comprises scrolling new keywords onto the user interface and scrolling old keywords off of the user interface as the mobile computing device receives new keywords from the audio and visual presentation system; and

in response to user input, scrolling old keywords back onto the user interface, to allow the user to utilize one of the old keywords to select a corresponding portion of the audio and visual program to be played by the audio and visual presentation system.

14. An article of manufacture according to claim 10, wherein the instructions enable the mobile device to perform further operations comprising:

after receiving at least one keyword for the audio and visual program from the audio and visual presentation system;

automatically retrieving, from a public wide area network, information relating to the at least one keyword; and

automatically displaying at least some of the information from the public wide area network relating to the at least one keyword.

15. An article of manufacture according to claim 14, wherein:

the operation of automatically retrieving information relating to the at least one keyword comprises;

automatically retrieving reference information about a meaning of the at least one keyword from an online encyclopedia; and

automatically retrieving purchasing information about products pertaining to the at least one keyword from a website of a vendor who offers said products for sale over the public wide area network; and

the operation of automatically displaying at least some of the information from the public wide area network relating to at least one of the keywords comprises simultaneously displaying at least some of the reference information and at least some of the purchasing information at the mobile computing device.

16. A mobile computing device comprising:

a wireless communications interface operable to receive, from an audio and visual presentation system, a stream of keyword data for an audio and visual program currently being played by the audio and visual presentation system, wherein the stream of keyword data comprises textual keywords that match spoken words from audio content of the audio and visual program;

a user interface module configured to automatically display two or more of the keywords in a user interface, and to receive user input indicating that a user has selected one of the keywords; and

a program control module configured to automatically cause the audio and visual presentation system to jump to a portion of the audio and visual program that contains the selected keyword, in response to the user input selecting the keyword.

17. A mobile computing device according to claim 16, wherein the keyword data comprises keywords extracted from closed caption data for the audio and visual program by the audio and visual presentation system.

18. A mobile computing device according to claim 16, wherein:

the keyword data further comprises time markers associated with corresponding keywords; and

the program control module is further configured to perform operations comprising;

generating a program control command instructing the audio and visual presentation system to jump to a selected portion of the audio and visual program, based on the selected keyword and the corresponding time marker; and

sending the program control command to the audio and visual presentation system via the wireless communications interface.

19. An audio and visual presentation device comprising:

a display device;

a decoder in communication with the display device, the decoder configured to display video content of an audio and visual program on the display device;

a keyword recognition module configured to automatically identify keywords from within textual data received by the audio and visual presentation device for the audio and visual program while the audio and visual presentation device is playing the audio and visual program, wherein the keywords comprises textual words that match spoken words from audio content of the audio and visual program;

a wireless communications interface operable to perform operations comprising;

while the audio and visual presentation device is playing the audio and visual program, sending, to a mobile device, a stream of keyword data for the audio and visual program, wherein the stream of keyword data comprises at least some of the keywords identified by the keyword recognition module; and

after sending at least some keyword data for the audio and visual program to the mobile device, receiving, from the mobile device, a program control command pertaining to a keyword selected by a user of the mobile device from the stream of keyword data sent to the mobile device; and

a program control module configured to automatically cause the audio and visual presentation device to shift the audio and visual program from a current play time of the audio and visual program to a different play time of the audio and visual program, based on the program control command from the mobile device.

20. An audio and visual presentation device according to claim 19, wherein the decoder comprises at least one component from the group consisting of:

a television tuner;

an optical disc player; and

a digital video recorder.

21. An audio and visual presentation device according to claim 19, wherein the decoder comprises at least one component from the group consisting of:

a television tuner;

an optical disc player;

a digital video recorder; and

a decoder application to execute on a microprocessor in the audio and visual presentation device.

22. An audio and visual presentation device according to claim 19, wherein the keyword recognition module is operable to automatically identify keywords by performing operations comprising:

automatically cross referencing the textual data received by the audio and visual presentation device for the audio and visual program with an existing database of important terms; and

automatically identifying as keywords words that appear both in the textual data for the audio and visual program and in the database of important terms.

23. An audio and visual presentation device according to claim 19, wherein the keyword recognition module is operable to automatically identify keywords by performing operations comprising:

automatically determining which words within the textual data are proper nouns; and

automatically identifying at least some of said proper nouns as keywords.

24. An audio and visual presentation device according to claim 19, wherein:

the program control command from the mobile device specifies a play time for the keyword selected by the user; and

the program control module is operable to automatically cause the audio and visual presentation device to jump from the current play time of the audio and visual program to the play time specified by the program control command.

25. An audio and visual presentation device according to claim 19, further comprising at least one item from the group consisting of:

a network interface operable to receive an encoded version of the audio and visual program from an external source; and

a storage medium in communication with the decoder, the storage medium operable to store at least part of the encoded version of the audio and visual program.