Spoken dialog system using prominence

• US 9,202,466 B2
• Filed: 03/18/2013
• Issued: 12/01/2015
• Est. Priority Date: 03/29/2012
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Heckmann, Martin
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/845,841
• Publication Number:
  US 20130262117A1
• Time in Patent Office: 988 Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

The invention presents a method for analyzing speech in a spoken dialog system, comprising the steps of: accepting an utterance by at least one means for accepting acoustical signals, in particular a microphone, analyzing the utterance and obtaining prosodic cues from the utterance using at least one processing engine, wherein the utterance is evaluated based on the prosodic cues to determine a prominence of parts of the utterance, and wherein the utterance is analyzed to detect at least one marker feature, e.g. a negative statement, indicative of the utterance containing at least one part to replace at least one part in a previous utterance, the part to be replaced in the previous utterance being determined based on the prominence determined for the parts of the previous utterance and the replacement parts being determined based on the prominence of the parts in the utterance, and wherein the previous utterance is evaluated with the replacement part(s).

Claims

1. A method for analyzing speech in a spoken dialog system, comprising the steps of:

accepting an utterance by at least one means for accepting acoustical signals, in particular a microphone,

analyzing, by a computer processor, the utterance and obtaining prosodic cues from the utterance using at least one processing engine,

wherein the utterance is evaluated based on the prosodic cues to determine a prominence of parts of the utterance, and wherein the utterance is analyzed to detect at least one marker feature indicative of the utterance containing at least one part to replace at least one part in a previous utterance, the part to be replaced in the previous utterance being determined based on the prominence determined for the parts of the previous utterance and the replacement parts being determined based on the prominence of the parts in the utterance, and wherein the previous utterance is evaluated with the replacement part(s).

2. The method of claim 1, wherein the utterance is a correction of the previous utterance, and wherein the utterance is a word or sentence.

3. The method of claim 1, wherein the prominence is determined based on a ranking.

4. The method of claim 1, wherein the prominence indicates a degree of importance of parts of an utterance, e.g. the emphasis a speaker sets on parts of the utterance.

5. The method of claim 1, wherein the marker feature is detected when at least parts of the previous utterance are repeated.

6. The method of claim 1, wherein the part(s) to be replaced in the previous utterance is/are used to improve a recognition accuracy by extracting at least one part with a pre-determined prominence from the utterance, e.g. the correction, extracting the part(s) to be replaced in the previous utterance, and comparing at least one recognition hypotheses for the extracted parts and inferring from this comparison a new recognition hypothesis for the part to be replaced of the previous utterance.

7. The method of claim 1, wherein the marker feature is determined by the prominence of the first part of the utterance either by itself or in combination with a lexical analysis of the utterance.

8. The method of claim 1, wherein the utterance is analyzed in form of a speech/acoustical and/or a video signal, and wherein the prosodic cues are either extracted from the speech/acoustical signal, the video signal, e.g. representing a recording of a user'"'"'s upper body, preferably including the head and face, or a combination of both.

9. The method of claim 1, wherein movements of the user'"'"'s head, facial muscles, mouth and/or eyebrows are used to determine the prosodic cues.

10. The method of claim 1, wherein a compressive transformation is applied to a mouth region, and wherein a tilt of a head and consequently the mouth region is corrected prior to applying the transformation, in particular a Discrete Cosine Transformation (DCT).

11. The method of claim 1, wherein a reliability for each of different information channels is calculated over which the prosodic cues are obtained, wherein an assignment of importance, i.e. prominence, on the different parts of the utterance is obtained by adaptively combining the different information channels considering previously calculated reliabilities, and wherein the reliability of the information channel is calculated based on illumination conditions.

12. A spoken dialog system, comprising

at least one means for accepting acoustical signals, in particular a microphone, for accepting an utterance,

at least one processing engine for analyzing, by a computer processor, the utterance and to obtain prosodic cues from the utterance,

wherein the processing engine is adapted to evaluate the utterance based on the prosodic cues to determine a prominence of parts of the utterance, and wherein the processing engine is adapted to analyze the utterance to detect at least one marker feature indicative of the utterance containing at least a part to replace at least a part in a previous utterance accepted by said acoustical signal means, the part to be replaced in the previous utterance being determined based on the prominence determined for the parts of the previous utterance and the replacement parts being determined based on the prominence of the parts in the utterance, and wherein the previous utterance is evaluated with the replacement part(s).

14. The system of claim 12, comprising a means for accepting visual signals for capturing a video signal.

13. A spoken dialog system, comprising:

at least one means for accepting acoustical signals, in particular a microphone, for accepting an utterance,

at least one processing engine for analyzing, by a computer processor, the utterance and to obtain prosodic cues from the utterance,

wherein the processing engine is adapted to evaluate the utterance based on the prosodic cues to determine a prominence of parts of the utterance, and wherein the processing engine is adapted to analyze the utterance to detect at least one marker feature indicative of the utterance containing at least a part to replace at least a part in a previous utterance accepted by said acoustical signal means, the part to be replaced in the previous utterance being determined based on the prominence determined for the parts of the previous utterance and the replacement parts being determined based on the prominence of the parts in the utterance, and wherein the previous utterance is evaluated with the replacement part(s); and

wherein the system is configured to perform a process comprising;

accepting the utterance by the at least one means for accepting acoustical signals, in particular the microphone,

analyzing the utterance and obtaining the prosodic cues from the utterance using the at least one processing engine,

wherein the utterance is evaluated based on the prosodic cues to determine the prominence of parts of the utterance, and wherein the utterance is analyzed to detect the at least one marker feature indicative of the utterance containing the at least one part to replace the at least one part in the previous utterance, the part to be replaced in the previous utterance being determined based on the prominence determined for the parts of the previous utterance and the replacement parts being determined based on the prominence of the parts in the utterance, and wherein the previous utterance is evaluated with the replacement part(s).