Sleep prevention dialog based car system

• US 6,236,968 B1
• Filed: 05/14/1998
• Issued: 05/22/2001
• Est. Priority Date: 05/14/1998
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kanevsky, Dimitri, Zadrozny, Wlodek Wlodzimierz
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): ARMSTRONG, ANGELA A
• Application Number:
  US09/078,807
• Time in Patent Office: 1,104 Days
• Field of Search: 704/270, 704/274, 704/275, 704/272, 704/257, 704/251, 704/258
• US Class Current: 704/275
• CPC Class Codes: A61B 5/4803 Speech analysis specially adapted for diagnostic purposes
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

An automatic dialog system capable of keeping a drive awake while driving during a long trip or one that extends into the late evening. The system carries on a conversation with the driver on various topics utilizing a natural dialog car system. The system includes an automatic speech recognition module, a speech generation module which includes speech synthesis or recorded speech, and possibly dynamically combined speech synthesizer and recorded speech, and a natural language processing module. The natural dialog car system analyzes a driver'"'"'s answer and the contents of the answer together with his voice patterns to determine if he is alert while driving. The system warns the driver or changes the topic of conversation if the system determines that the driver is about to fall asleep. The system may also detect whether a driver is effected by alcohol or drugs.

Claims

1. A dialog car system comprising:

a speech recognition circuit, operative to receive words an sounds uttered by a drive of a vehicle, and to produce decoded voice signals corresponding to said words and sounds;

a natural language processor coupled to receive said decoded voice signals output by speech recognition circuit, and being operative to process and produce responses based on said decoded voice signals;

a speech generator, responsive to an output derived from said natural language processor, for generating speech for the driver to hear so as to carry on a dialog with the driver;

an analyzer, coupled to said natural language processor and said speech recognition circuit, for analyzing the words and sounds uttered by the drive and to determine whether the drive is alert or about to fall asleep; and

a conversational planner, coupled to an output of said driver analyzer for planning the type and content of speech to generate by said speech generator, wherein said conversational planner comprises means for receiving indications of conditions of the driver, means for continuing the dialog when said conditions of the driver are within an acceptable level, and means for changing the dialog when at least one of said conditions of the driver is outside an acceptable level.

2. The dialog car system as defined by claim 1, further comprising:

an intelligent text processor, coupled to an input of said conversational planner, for producing various types of dialog to be considered by said conversational planner to be included in the dialog with the driver; and

a car media, coupled to said intelligent text processor, being operative to produce audio programming information based on a plurality of sources.

3. The dialog car system as defined by claim 2, wherein said car media comprises means for receiving programming information from a plurality of sources;

said system further comprising means for recognizing a certain word or series of words in said programming information, to allow predefined audio programming to be selected and transferred to said conversational planner.

4. The dialog car system as defined by claim 3, wherein said plurality of sources includes one or more of audio tapes, audio books, CD player units, DVD player units, telephonic communications, radio, sites on the internet, and remote audio input.

5. The dialog car system as defined by claim 1, further comprising an alarm, coupled to an output of said conversational planner, and being activated to produce an alarm when said conversational planner detects that the driver is about to fall asleep, wherein said alarm includes at least one of the following:

an audio alarm, a visual alarm, and physically impacting the driver.

6. The dialog car system as defined by claim 1, further comprising a driver profile module for storing information particular to the driver, wherein said conversational planner is responsive to said information stored in said driver profile module for planning the type and content of dialog with the driver.

7. The dialog car system as defined by claim 1, further comprising a language constrain module coupled between said speech recognition circuit and speech generator, and wherein said language constrain module dynamically constrains said speech recognition circuit to produce said decoded voice signals restricted to predetermined topics of conversation.

8. The dialog car system as defined by claim 1, wherein said natural language processor comprises:

a parsing module which receives said decoded voice signals including text data from said speech recognition circuit, for parsing structure of said text data to produce structural descriptions of said text;

a semantic processor, coupled to an output of said parsing module, for interpreting semantic content of each phrase output by said parsing module;

a situation module, coupled to an output of said semantic processor, for relating an output of said semantic processor to one of a plurality of situations stored in a first memory module;

a formal text module responsive to an output of said semantic processor for generating a formal representation of phrases, including logical structures and correlations between verbs and nouns;

a second memory module for storing a list of operations that can be performed with formal sentences from said formal text module;

a third memory module for storing a list of possible replies to decoded voice signals received from the driver;

a connectivity graph module for connecting words with paths if two words are met in the same sentence or have an association of words; and

a decision tree coupled to said situation module;

formal text module;

second memory module, third memory module, and connectivity graph module, for processing and producing meaningful responses, and outputting said meaningful responses to said conversational planner.

9. The dialog car system as defined by claim 1, wherein said driver analyzer comprises:

input means for receiving an input data stream, including voice and text data;

a time evaluator, coupled to said input means, for time stamping said voice and text data corresponding to when the driver receives a voice prompt and when the driver answers, and for evaluating the length of time to respond to an inquiry;

an answer evaluator, coupled to said input means, for evaluating said input data stream, determining whether said input data stream matches a predetermined set of possible answers, and grading said input data stream on a scale; and

a voice pattern evaluator, coupled to said input means, for matching said input data stream with a predetermined set of voice patterns to detect one or more characteristics of said input data stream.

10. The dialog car system as defined by claim 1, wherein said conversational planner comprises:

an integrated evaluator, responsive to outputs of said driver analyzer, for classifying responses of the driver into different categories depending on predetermined characteristics of the response, and determining a state of condition of the driver.

11. The dialog car system as defined by claim 10, wherein said conversational planner comprises means for determining whether to continue dialog in a normal manner, means for determining whether a response from the driver was at least one of good, not very good, bad, or not understood.

12. A dialog car system comprising:

a speech recognition circuit, operative to receive words and sounds uttered by a driver of a vehicle, and to produce decoded voice signals corresponding to said words and sounds;

a natural language processor, coupled to receive said decoded voice signals output by said speech recognition circuit, and being operative to process and produce responses based on said decoded voice signals;

a driver analyzer, coupled to said natural language processor and said speech recognition circuit, for analyzing said decoded voice signals and to determine whether the driver is alert or about to fall asleep;

a driver profile module for storing information particular to the driver;

a car media operative to produce programming information from a plurality of sources;

an intelligent text processor, coupled to an output of said car media, for producing various types of dialog to be considered by a conversational planner to be included in the dialog with the driver; and

said conversational planner, responsive to said information stored in said driver profile module and coupled to an output of said driver analyzer and an output of said intelligent text processor, for planning the type and content of speech based on input received from said driver analyzer and intelligent text processor;

a speech generator, responsive to an output derived from said conversational planner, for generating speech for the driver to hear to carry on a dialog with the driver; and

an external service provider, coupled to said car media, driver profile and conversational planner in order to upgrade said car media, driver profile and conversational planner with new data, including type and content of programming information.

13. The dialog car system as defined by claim 12, further comprising an adapter for coupling said external service provider to said car media, driver analyzer, driver profile and conversational planner module.

14. The dialog car system as defined by claim 12, further comprising an alarm, coupled to an output of said conversational planner, and being activated to produce an alarm when said conversational planner detects that the driver is about to fall asleep, wherein said alarm includes at least one of the following:

an audio alarm, a visual alarm, and physically impacting the driver.

15. The dialog car system as defined by claim 13, wherein said external service provider receives alert signals from said analyzer if it is determined that the driver is not responding properly to activation of said alarm.

16. The dialog car system as defined in claim 15, wherein said external service provider evaluates the success rates of various types and contents of programming information, and updates the dialog car system to implement programming information that is successful in preventing a driver from falling asleep.