Automatic control of household activity using speech recognition and natural language

• US 20010041980A1
• Filed: 06/06/2001
• Published: 11/15/2001
• Est. Priority Date: 08/26/1999
• Status: Active Grant

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Howard, John Howard K., Junqua, Jean-Claude
• Granted Patent: US 6,513,006 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications

Abstract

Speech recognition and natural language parsing components are used to extract the meaning of the user'"'"'s spoken input. The system stores a semantic representation of an electronic activity guide, and the contents of the guide can be mapped into the grammars used by the natural language parser. Thus, when the user wishes to navigate through the complex menu structure of the electronic activity guide, he or she only needs to speak in natural language sentences. The system automatically filters the contents of the guide and supplies the user with on-screen display or synthesized speech responses to the user'"'"'s request. The system allows the user to communicate in a natural way with a variety of devices communicating with the home network or home gateway.

Claims

1. An information filtering system for a home network having at least one home component communicating therewith, comprising:

a speech recognizer having parser for extracting semantic information from a user'"'"'s input speech;

a data store containing at least one electronic activity guide;

a dialog manager for filtering said stored electronic activity guide based on said semantic information and for providing filtered information associated with said home component.

2. The system of

claim 1

wherein said home component is selected from the group consisting of audio/video equipment, appliances, computers, home automation systems, fire alarm systems, burglar alarm systems, telephone systems, intercom systems, gateway systems and internet access systems.

3. The system of

claim 1

wherein said dialog manager provides said filtered information to the user in audible or visual form.

4. The system of

claim 1

further comprising a mapping module coupled to said data store for updating the contents of said data store from a source of electronic activity guide information.

5. The system of

claim 1

further comprising a home gateway module that couples said network to the internet and wherein said filtering system filters information obtained via the internet based on said semantic information.

6. The system of

claim 1

wherein a plurality of home components are in communication with said network and wherein said filtering system includes a plurality of electronic activity guides corresponding to said plurality of home components.

7. The system of

claim 1

wherein said home component includes an associated microphone through which said user provides said input speech.

8. The system of

claim 1

wherein said home component includes an associated display by which said dialog manager provides said filtered information.

9. The system of

claim 1

wherein said dialog manager includes a speech synthesizer for providing the user with synthesized speech information.

10. The system of

claim 1

wherein said dialog manager includes a speech synthesizer for providing the user with said filtered information as synthesized speech.

11. The system of

claim 1

wherein said parser is a goal-oriented parser defining a plurality of goal-oriented frames having slots corresponding to information represented in said electronic activity guide.

12. The system of

claim 1

wherein said parser is a natural language parser having a set of predefined grammars that correspond to information represented in said electronic activity guide.

13. The system of

claim 4

wherein said mapping module includes a mapping module updating system for updating the functionality of said mapping module based on information obtained from a remote site.

14. The system of

claim 1

wherein said dialog manager includes a user profile data store for storing a representation of a user'"'"'s past use of said filtering system, and wherein said dialog manager uses said profile data store to further filter said stored electronic activity guide representation.

15. The system of

claim 1

further comprising user profile data store and speaker verification system for accessing said user profile data store.

16. The system of

claim 1

further comprising user profile data store and speaker identification system for accessing said user profile data store.