Retrieval of additional information not found in interactive TV or telephony signal by application using dynamically extracted vocabulary

• US 5,809,471 A
• Filed: 03/07/1996
• Issued: 09/15/1998
• Est. Priority Date: 03/07/1996
• Status: Expired due to Term

• Current Assignee: MediaTek, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Brodsky, Marc Herbert
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US08/612,210
• Time in Patent Office: 922 Days
• Field of Search: 395/2.84, 348/13, 348/460, 348/468, 704/10, 704/275, 379/93.25
• US Class Current: 704/275
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: H04H 20/28 Arrangements for simultaneous broadcast of plural pieces of information
• CPC Class Codes: H04H 20/38 Arrangements for distribution where lower stations, e.g. receivers, interact with the broadcast
• CPC Class Codes: H04H 2201/30 characterised by the use of a return channel, e.g. for collecting users' opinions, for returning broadcast space/time information or for requesting data
• CPC Class Codes: H04H 2201/37 via a different channel
• CPC Class Codes: H04H 60/48 for recognising items expressed in broadcast information
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/65 for using the result on users' side
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04N 21/4112 having fewer capabilities than the client, e.g. thin client having less processing power or no tuning capabilities
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/488 Data services, e.g. news ticker systems specially adapted for using meteorological information in broadcast systems H04H60/71
• CPC Class Codes: H04N 7/0882 for the transmission of character code signals, e.g. for teletext circuits for the digital non-picture data signal H04N7/035
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles
• CPC Class Codes: H04N 7/17363 at or near the user terminal

Abstract

A method and apparatus is provided to enable a user watching and/or listening to a program to retrieve supplemental information or supplemental entertainment pertaining to an item just seen or words just heard. The apparatus includes a recognition system which recognizes the user'"'"'s request and causes its satisfaction to be provided. A dynamically changing dictionary is created from items or keywords extracted from the most recently received program portion. The items or keywords may be contained in closed caption text in a TV signal. Particular items or keywords may be assigned a high priority so that they are maintained in the dictionary for a long duration. The user'"'"'s request is matched against this continually changing dictionary. Upon recognition of the request, a search is initiated to access, import and deliver to the user the information required to satisfy the request. The information may be presented to the user separately or in coincidence with the normal program using picture in picture or split screen. Alternate embodiments enable preprocessing to obtain and store expanded information for such items and keywords deemed to be of high priority. This speeds up the satisfaction of the user'"'"'s request for these items or keywords.

Claims

1. A system receiving an input signal from a program and displaying said program on a program display medium to a user, said input signal having a plurality of information components for which an application can provide expanded information not found provided in said input signal, said system comprising:

a context extractor for extracting each of said plurality of information components from said input signal;

a buffer for receiving and storing each of said plurality of information components extracted by said context extractor for a predetermined time;

a user interface for said user to make a request to receive said expanded information regarding at least one of said plurality of information components; and

a recognition processor for receiving said request, for recognizing a match between said request and said information components extracted by said context extractor, and for forwarding said match to said application, whereby said application retrieves said expanded information and provides said expanded information for display on said displaying medium.

2. A system as in claim 1 further comprising an application medium to provide said expanded information for at least one of said plurality of information components.

3. A system as in claim 1 wherein said recognition processor performs preprocessing on at least one of said plurality of information components extracted.

4. A system as in claim 1 wherein said request is a voice request made by said user.

5. A system as in claim 1 wherein said buffer has a storage capacity, and where upon said received information components filling said storage capacity so that additionally received said information components are stored by overwriting the oldest stored said information components.

6. A system as in claim 1 wherein said plurality of information components extracted from said input signal form a dynamic dictionary.

7. A system as in claim 1 wherein said input signal includes closed caption text, and wherein said extractor includes a means to separate said closed caption text from said input signal, and wherein at least one of said plurality of information components is included in said closed caption text.

8. A system as in claim 7 wherein said input signal is an NTSC type TV signal and where said closed caption text is on NTSC line 21.

9. A system as in claim 1 further comprising a means for assigning a priority to each of said information components extracted by said context extractor.

10. A system as in claim 9 wherein each of said information components extracted is stored in said buffer for at least a predetermined duration and wherein said duration is dependent upon said priority assigned to each of said extracted information components.

11. A system as in claim 1 wherein said expanded information is provided together with said program display using at least one means selected from the group consisting of picture in picture, split screen, video text and selective picture.

12. A real time recognition system comprising:

means for extraction of a target vocabulary from a current stream of information;

a buffer to store said target vocabulary as said target vocabulary is updated;

means for a user who views said current stream of information to make a request for additional information about an entry in said target vocabulary extracted from said current stream; and

a recognition processor that converts said user'"'"'s request into a matched topic against said target vocabulary.

13. The system of claim 12 wherein said current stream has a format for viewing, and wherein said matched topic is recognizable by an application, said system further comprising:

means for relaying said matched topic to said application;

means for retrieving said additional information from said application; and

means for forwarding an amount of said additional information back to said user via a retrieved signal stream in said format for viewing.

14. The system of claim 12 further comprising means for said user to display at least a part of the buffer contents so that said user can choose a valid request.

15. A dynamic dictionary comprising:

an input information stream having a primary stream and a secondary stream, said secondary stream comprising a plurality of recognizable information components;

a recognizer for receiving said information stream, recognizing and selecting said plurality of recognizable information components; and

a buffer for storing said plurality of recognizable information components selected by said recognizer for a predetermined time.

16. A dictionary as in claim 15 further comprising an index of said plurality of recognizable information components selected by said recognizer.

17. A dictionary as in claim 16 wherein said index is sorted in an ascending alpha numeric sequence.

18. A dictionary as in claim 16 further comprising a menu display means for displaying said index.

19. A dictionary as in claim 15 wherein said buffer has a capacity, each of said plurality of recognizable information components is stored for at least a predetermined duration, and a plurality of first received information components are being stored for a longer duration than a later received plurality of information components, and where upon said capacity becomes filled with said plurality of recognizable information components, additionally received said plurality of recognizable information components are stored by overwriting said plurality of first received information components before overwriting said later received plurality of information components.

20. A real time user voice recognition system interacting with an application, said application being capable of providing additional information for a plurality of topics, said system comprising:

means for extracting an updated target vocabulary from a current stream of information having a format for viewing;

a buffer for receiving said updated target vocabulary as said updated target vocabulary is extracted and for storing said updated target vocabulary for a predetermined time;

means for said user viewing said current stream of information to make a request for an amount of additional information, not provided in said current stream of information, about at least one entry of said updated target vocabulary recently referred to in said stream of information;

a recognition processor for converting said request into a matched topics against said updated target vocabulary, and for relaying said matched topic to said application;

means for retrieving said amount of additional information from said plurality of topics of said application; and

means for sending said amount of additional information to said user via a retrieved signal stream in said format for viewing.

21. A recognition system as in claim 20 further comprised of a database application capable of providing additional information for at least one of said plurality of topics.

22. A recognition system as in claim 20 wherein said recognition processor performs preprocessing for at least one entry of said updated target vocabulary.

23. A recognition method for receiving an input signal form a program and displaying said program on a program display medium to a user, said input signal having a plurality of information components for which an application can provide expanded information not provided in said input signal, said method comprising the steps of:

extracting each of said plurality of information components from said input signal;

storing in a buffer each of said plurality of information components extracted for a predetermined time;

responding to a request from said user to receive said expanded information regarding at least one of said plurality of information components extracted;

matching said request to said information components extracted; and

forwarding said match to said application whereupon said application provides said expanded information for display on said displaying medium.

24. A method as in claim 23 further comprising the step of providing an application medium to provide said expanded information for at least one of said each of plurality of information components.

25. A method as in claim 23 further comprising a step of preprocessing at least one of said extracted information components.

26. A method as in claim 23 wherein said request is made by and recognized from a voice request by said user.

27. A method as in claim 23 wherein said buffer has a storage capacity, and where upon said received information components fills said storage capacity such that additionally received said information components are stored by overwriting the oldest stored said information components.

28. A method as in claim 23 further comprising a step of forming a dynamic dictionary of said information components of said program.

29. A method as in claim 23 further comprising the step of assigning a priority to each of said extracted information components and wherein each of said extracted information components is stored for a particular duration and wherein said particular duration is dependent upon said assigned priority.

30. A method for forming a real time recognition system that comprises the steps of:

extracting a target vocabulary from a current stream of information;

receiving said target vocabulary in a buffer as said target vocabulary is updated and storing said target vocabulary in said buffer for a predetermined time;

responding to a current user'"'"'s request for additional information not provided in said stream of information about an entry in said target vocabulary; and

matching said user'"'"'s request against said buffered vocabulary.

31. A method of claim 30 wherein said current stream is in a format for viewing, said method further comprising the steps of relaying said matched topic to an application that retrieves said request for said additional information, and forwarding an amount of retrieved said additional information back to said user via a retrieved signal stream in said format for viewing.

32. A method as in claim 30 further comprising the step of displaying said additional information together with said current stream using at least one means selected from the group consisting of picture in picture, split screen, video text and selective picture.

33. A method of claim 30 further comprising the step of displaying at least a part of said buffer contents so that said user can choose a valid request.

34. A method as in claim 33 further comprising the step of displaying said selected information in a menu.

35. A method for forming a dynamic dictionary comprising the steps of:

providing an input information stream having a first stream and a second stream, said second stream comprising a plurality of recognizable information components;

recognizing and selecting said information components; and

storing said selected information components in a buffer for a predetermined time.

36. A method as in claim 35 further comprising the step of indexing said selected information components.

37. A method as in claim 36 wherein said index is sorted in an ascending alpha numeric sequence.