Method and system for interactive conversational dialogue for cognitively overloaded device users

• US 20060074670A1
• Filed: 09/27/2004
• Published: 04/06/2006
• Est. Priority Date: 09/27/2004
• Status: Active Grant

• Current Assignee: Robert Bosch Corporation (Robert Bosch GmbH), Volkswagen Of America (Volkswagen AG)
• Original Assignee: Robert Bosch Corporation (Robert Bosch GmbH), Volkswagen Of America (Volkswagen AG)
• Inventors: Raghunathan, Badri, Schmidt, Hauke, Weng, Fuliang, Cavedon, Lawrence, Mirkovic, Danilo, Gruenstein, Alexander, Hiatt, Laura, Peters, Stanley
• Granted Patent: US 7,716,056 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A system and method to interactively converse with a cognitively overloaded user of a device, includes maintaining a knowledge base of information regarding the device and a domain, organizing the information in at least one of a relational manner and an ontological manner, receiving speech from the user, converting the speech into a word sequence, recognizing a partial proper name in the word sequence, identifying meaning structures from the word sequence using a model of the domain information, adjusting a boundary of the partial proper names to enhance an accuracy of the meaning structures, interpreting the meaning structures in a context of the conversation with the cognitively overloaded user using the knowledge base, selecting a content for a response to the cognitively overloaded user, generating the response based on the selected content, the context of the conversation, and grammatical rules, and synthesizing speech wave forms for the response.

Claims

1. A dialog system, comprising:

a knowledge base to store information regarding a device and a domain, and to categorize the information in at least one of a relational and a ontological manner;

a device/application manager to isolate and encapsulate the device information for a plug-n-play of the domain;

a speech recognition module to receive speech from a user and convert the speech into a word sequence, and to recognize at least one of full and partial proper names;

a natural language understanding module to identify meaning structures from the word sequence using a statistically dependent model of the domain, and to adjust a boundary of the at least one of the full and partial proper names;

a dialogue management module to interpret the meaning structures in a context of the conversation using the device information, and to construct a content of a response;

a response generation module to select content and generate the response based on the context and grammatical rules; and

a speech synthesis module to synthesize speech wave forms for the response.

2. The dialog system of claim 1, wherein the device includes at least one of a mobile telephone, a navigation system, a point of interest service, an audio device, an in-car device, and an MP3 player.

3. The dialog system of claim 1, further comprising:

a speech enhancer to provide noise reduction and echo cancellation of the received speech.

4. The dialog system of claim 1, further comprising:

a prosody detection module to detect at least one of intonation, pause, and duration of the received speech.

5. The dialog system of claim 1, wherein the dialog system is distributed to allow dynamic registration of at least one of a new module and a new device.

6. The dialog system of claim 1, wherein the natural language understanding module includes a statistical tagger to assign part of speech tags to the word sequence, a statistical dependency parser to form dependency relations using conditional probability and mutual information, and a feature mapper to an Extensible Markup Language representation of the word sequence using the dependency relations.

7. The dialog system of claim 1, wherein the dialog management module is configured to support a multi-domain environment.

8. The dialog system of claim 1, wherein the dialog management module is configured to manage multi-thread conversations.

9. The dialog system of claim 1, wherein the dialog management module is configured to support a device-specific activity model.

10. The dialog system of claim 1, wherein the dialog management module includes a semantic analyzer.

11. The dialog system of claim 1, wherein the dialog management module is configured to support a dialog move tree.

12. The dialog system of claim 1, wherein the dialog management module is configured to support multi-topic conversations.

13. The dialog system of claim 1, wherein the dialog system at least one of adjusts and verifies the partial proper names.

14. A method of interactively conversing with a cognitively overloaded user of a device, the method comprising:

maintaining a knowledge base of information regarding the device and a domain;

organizing the information in at least one of a relational manner and an ontological manner;

receiving speech from the user;

converting the speech into a word sequence;

recognizing at least one of a full and partial proper name in the word sequence;

identifying meaning structures from the word sequence using a model of the domain information;

adjusting a boundary of the at least one of the full and partial proper names to enhance an accuracy of the meaning structures;

interpreting the meaning structures in a context of the conversation with the cognitively overloaded user using the knowledge base;

selecting a content for a response to the cognitively overloaded user;

generating the response based on the selected content, the context of the conversation, and grammatical rules; and

synthesizing speech wave forms for the response.

15. The method of claim 14, further comprising:

at least one of;

reducing a noise of the received speech; and

canceling an echo of the received speech.

16. The method of claim 14, further comprising:

registering the device to advertise a capability of the device in linguistic semantic form; and

selecting the device based on at least one of the capability of the device in linguistic semantic form and the context of the conversation, for all types of user requests.

17. The method of claim 14, further comprising:

identifying at least one of a pause, hesitation, and duration in the received speech.

18. The method of claim 14, further comprising:

forming a proper named class in a class-based grammar by training n-gram models on the named set only; and

dynamic inserting the trained n-gram models in the class-based grammar as the proper name class.

19. The method of claim 14, wherein the identifying of the meaning structures further includes:

identifying a part of speech tag for each word of the word sequence;

deriving dependency structures from the word sequence to dynamically adapt the model of the domain information;

converting the dependency structures to feature structures; and

associating the part of speech tag and features structures with a semantic topic of the word sequence.

20. A storage medium having a set of instructions executable by a processor to perform the following:

maintaining a knowledge base of information regarding a device and a domain;

organizing the information in at least one of a relational manner and an ontological manner;

receiving speech from a user of the device;

converting the speech into a word sequence;

recognizing a partial proper name in the word sequence;

identifying meaning structures from the word sequence using a model of the domain information;

adjusting a boundary of the partial proper names to enhance an accuracy of the meaning structures;

interpreting the meaning structures in a context of the conversation with the user using the knowledge base;

selecting a content for a response to the user;

generating the response based on the selected content, the context of the conversation, and grammatical rules; and

synthesizing speech wave forms for the response.