Supervised contextual language acquisition system

• US 5,748,841 A
• Filed: 04/10/1997
• Issued: 05/05/1998
• Est. Priority Date: 02/25/1994
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Corporation Of America (Panasonic Holdings Corporation)
• Original Assignee: Jean-Claude Junqua, Philippe Morin
• Inventors: Morin, Philippe, Junqua, Jean-claude
• Primary Examiner(s): Knepper, David D.
• Assistant Examiner(s): Dorvil, Richemond
• Application Number:
  US08/833,912
• Time in Patent Office: 390 Days
• Field of Search: 395/2.66, 395/2.86, 395/751, 395/759, 395/2.64
• US Class Current: 704/257
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

The language acquisition system assists the user in acquiring the language of an application. The system uses the dialogue context, a dialogue model and syntactic-semantic grammars to progressively build commands which, to the application program, are syntactically and semantically correct in the current context and which can be interpreted by the dialogue server which then controls the application program. The system is independent of any particular application language. The system is also multimodal and supports both speech and text input. A toolkit is provided to add this functionality to virtually any application program.

Claims

1. A supervised contextual restricted natural language acquisition system for computerized applications, comprising:

first means for defining and storing a dialogue history context;

second means for defining and storing a dialogue model;

third means for defining and storing at least one syntactic-semantic grammar;

fourth means responsive to said first, second and third means for building a language to individually represent both dialogue-specific expressions and application specific application expressions and means to represent said application-specific expressions as at least one prediction tree representing at least one possible dialogue that is semantically consistent with the stored dialogue history context and stored dialogue model and fifth means to interpret said dialogue-specific expressions to supply instructions to the language acquisition system; and

user interface means for providing language assistance to the user based on said prediction tree.

2. The system of claim 1 wherein said third means includes means for defining and storing at least one application-specific grammar and at least one system-specific grammar.

3. The system of claim 1 wherein said second means includes means for defining and storing a dialogue model which defines the type of interaction in terms of dialogue answers.

4. The system of claim 1 wherein said third means includes a database of grammar rules arranged as uniquely labeled rules.

5. The system of claim 1 in which the restricted natural language is used to communicate with a target application and wherein said third means includes a database of scripts comprising possible dialogue scenarios supported by said a target application.

6. The system of claim 1 in which the restricted natural language is used to communicate with a target application and further comprising means for establishing a communication channel between a user and said target application.

7. The system of claim 6 wherein said first means includes a mechanism for monitoring the communication channel and for making a record of the context state of the target application during interaction with the application by the user.

8. The system of claim 6 wherein said fourth means has mechanism for monitoring the communication channel and for rebuilding at least one prediction tree automatically as a user interacts with the target application over said communication channel.

9. The system of claim 6 further comprising multi-modal communication support mechanism coupled to said communication channel whereby the user may communicate with the application using at least voice and manual input modes.

10. The system of claim 1 wherein said first means includes a mechanism for recording the context state of the language acquisition system as a user interacts with it through said user interface.

11. The system of claim 1 wherein said user interface includes multi-modal input mechanism whereby the user may communicate with the language acquisition system using at least voice and manual input modes.

12. An interactive restricted natural language acquisition system for a computerized application comprising:

an application manager coupled to said computerized application for providing an interface to said computerized application;

at least one dialogue database containing syntactic and semantic information regarding a language;

an input/output manager having at least one generator for generating messages using said dialogue database and having at least one recognizer for processing user input using said dialogue database and for extracting semantic information from said user input;

a dialogue manager coupled to said application manager and to said input/output manager for interpreting said semantic information and for selectively issuing commands through said application manager to the computerized application in response to said semantic information;

said dialogue manager including means for selectively building first and second tree structures based on current and historical user interaction said tree structures representing dialogue-specific and application-specific information respectively whereby the user is provided with language acquisition assistance on an interactive basis.

13. The system of claim 12 wherein said system includes user actuable means for invoking the building of said tree structure, whereby language acquisition assistance is activated at the user'"'"'s initiative.

14. The system of claim 12 wherein said dialogue manager includes context handler for maintaining a first record of a short-term context and a second record of a long-term context, said context handler automatically updating said first and second records in response to interaction by the user.

15. The system of claim 12 wherein said input/output manager comprises a multimodal interface supporting at least text and speech input.

16. The system of claim 12 wherein said input/output manager comprises a multimodal interface supporting at least text and speech output.

17. The system of claim 12 wherein said dialogue manager includes a dialogue processor for defining a dialogue model which supports both meta-commands and application commands used for the interaction.

18. The system of claim 12 wherein said application manager comprises a toolkit of communications functions for enabling the exchange of information between said computerized application and said dialogue server.

19. The system of claim 12 wherein said dialogue manager includes means for selectively building an ordered list arranged in order of a predetermined plausibility.

20. The system of claim 12 wherein said application manager communicates with said dialogue manager through application scripts that describe the application scenario at the dialogue level.

21. The system of claim 12 wherein said dialogue manager has a speech mode for handling speech input consisting of words and sequences of words.

22. The system of claim 12 wherein said dialogue manager has a text mode for handling text input consisting of characters, words and sequences of words.

23. The system of claim 12 wherein said input/output manager comprises a multimodal interface supporting mouse input.

24. The system of claim 12 wherein said input/output manager comprises a multimodal interface supporting touch screen input.

25. A language acquisition system to assist a user in a dialogue with a computer-implemented application program, comprising:

input system for supporting at least one of text and speech modes of input;

a first means for storing a dialogue context;

a second means for defining a dialogue model which describes the structure of a dialogue;

a third means for defining at least one syntactic-semantic grammar;

fourth means coupled to said first, second and third means to interactively assist the user in building commands which to the application program are syntactically and semantically correct and which can be interpreted by the dialogue manager in the current context;

said fourth means for generating assistance differently based on the mode of input supplied to said input system.

26. The system of claim 25 wherein the fourth means generates assistance selectively at a character-by-character, word-by-word or phrase-by-phrase basis depending on the mode of input.

27. The language acquisition system of claim 25 wherein said third means defines a restricted natural language grammar.

28. The language acquisition system of claim 25 further comprising means for supplying actions resulting from said commands to said application program.

29. The language acquisition system of claim 28 further comprising user selectable means for selectively building said commands and controlling whether said commands will be executed by said application program or not.

30. The language acquisition system of claim 25 wherein said first, second and third means are interactively coupled together to build said commands progressively.

31. The language acquisition system of claim 30 wherein said commands are built progressively in accordance with at least one syntactic-semantic grammar such that user errors in syntax are automatically detected.

32. The language acquisition system of claim 30 wherein said commands are built progressively in accordance with at least one syntactic-semantic grammar such that user errors in semantics are automatically detected.

33. The language acquisition system of claim 25 wherein a semantic representation defined in the syntactic-semantic grammar renders said language acquisition system independent of the application program language.

34. The language acquisition system of claim 25 wherein said third means is constructed to receive data describing at least one syntactic-semantic grammar and for establishing a set of rules based on said data for building said commands, whereby the language acquisition system can be adapted to work with a plurality of different languages.

35. The language acquisition system of claim 25 wherein said third means is constructed to receive data describing at least one syntactic-semantic grammar and for establishing a set of rules based on said data for building said commands, whereby the language acquisition system can be adapted to work with a plurality modes of communication, including text and speech.

36. The language acquisition system of claim 25 wherein said first, second and third means are configured to define a first mechanism which is independent of any application program and to define a second mechanism which is dependent on a particular application program.

37. The language acquisition system of claim 25 wherein said dialogue model supports a plurality of different types of dialogues including (a) negation of a prior dialogue;

(b) direct answer; and

(c) reference to a prior dialogue.