Automatic control of household activity using speech recognition and natural language

• US 6,513,006 B2
• Filed: 06/06/2001
• Issued: 01/28/2003
• Est. Priority Date: 08/26/1999
• Status: Expired due to Term

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Howard, John, Junqua, Jean-claude
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Nolan, Daniel
• Application Number:
  US09/875,740
• Publication Number:
  US 20010041980A1
• Time in Patent Office: 601 Days
• Field of Search: 704/270.1, 704/275, 704/246
• US Class Current: 704/257
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications

Abstract

Speech recognition and natural language parsing components are used to extract the meaning of the user'"'"'s spoken input. The system stores a semantic representation of an electronic activity guide, and the contents of the guide can be mapped into the grammars used by the natural language parser. Thus, when the user wishes to navigate through the complex menu structure of the electronic activity guide, he or she only needs to speak in natural language sentences. The system automatically filters the contents of the guide and supplies the user with on-screen display or synthesized speech responses to the user'"'"'s request. The system allows the user to communicate in a natural way with a variety of devices communicating with the home network or home gateway.

Claims

1. An information filtering system for a home network having at least one home components communicating therewith, comprising:

a speech recognizer for processing speech input;

a data store containing at least two electronic activity guides relating to different home component;

a natural language parser in communication with said speech recognizer and operable to extract semantic information from the speech input, the natural language parser having a set of grammars that correlate to contents of at least one electronic guide associated with a first home component;

a guide mapping module having access to the data store and operable to update the set of grammars with contents of an electronic activity guide associated with a second home component; and

a dialog manager for filtering said electronic activity guides based on said semantic information and for providing a user with information regarding available activities in response to the user'"'"'s input speech.

2. The system of claim 1 wherein said home component is selected from the group consisting of audio/video equipment, appliances, computers, home automation systems, fire alarm systems, burglar alarm systems, telephone systems, intercom systems, gateway systems and internet access systems.

3. The system of claim 1 wherein said dialog manager provides said filtered information to the user in audible or visual form.

4. The system of claim 1 further comprising a mapping module coupled to said data store for updating the contents of said data store from a source of electronic activity guide information.

5. The system of claim 1 further comprising a home gateway module that couples said network to the internet and wherein said filtering system filters information obtained via the internet based on said semantic information.

6. The system of claim 1 wherein a plurality of home components are in communication with said network and wherein said filtering system includes a plurality of electronic activity guides corresponding to said plurality of home components.

7. The system of claim 1 wherein said home component includes an associated microphone through which said user provides said input speech.

8. The system of claim 1 wherein said home component includes an associated display by which said dialog manager provides said filtered information.

9. The system of claim 1 wherein said dialog manager includes a speech synthesizer for providing the user with synthesized speech information.

10. The system of claim 1 wherein said dialog manager includes a speech synthesizer for providing the user with said filtered information as synthesized speech.

11. The system of claim 1 wherein said parser is a goal-oriented parser defining a plurality of goal-oriented frames having slots corresponding to information represented in said electronic activity guide.

12. The system of claim 1 wherein said parser is a natural language parser having a set of predefined grammars that correspond to information represented in said electronic activity guide.

13. The system of claim 4 wherein said mapping module includes a mapping module updating system for updating the functionality of said mapping module based on information obtained from a remote site.

14. The system of claim 1 wherein said dialog manager includes a user profile data store for storing a representation of a user'"'"'s past use of said filtering system, and wherein said dialog manager uses said profile data store to further filter said stored electronic activity guide representation.

15. The system of claim 1 further comprising user profile data store and speaker verification system for accessing said user profile data store.

16. The system of claim 1 further comprising user profile data store and speaker identification system for accessing said user profile data store.

17. An information filtering system for use with a home network having at least one home component communicating therewith, the system comprising:

a speech recognizer having a natural language parser for extracting semantic information from a user'"'"'s request, the parser comprising;

a) a global parser operable to extract overall semantics of the user'"'"'s request; and

b) a local parser operable to provide parse hypotheses, said local parser operable according to a local parsing method, the method comprising;

i) performing a lexical analysis;

ii) performing parallel parse-forest generation; and

iii) performing analysis and synthesis of parsed components;

a guide mapping module operable to construct a semantic representation of an electronic activity guide based on a source of electronic activity guide information, the electronic activity guide having available activities relating to said home component;

a stored semantic representation of the electronic activity guide, the stored semantic representation of the electronic activity guide stored in memory accessible to the natural language parser; and

a dialog manager operable to filter the stored semantic representation of the electronic activity guide based on the semantic information, the dialog manager further operable to provide a user with information regarding available activities in response to the user'"'"'s request.