Speech recognition dependent on text message content

• US 9,202,465 B2
• Filed: 03/25/2011
• Issued: 12/01/2015
• Est. Priority Date: 03/25/2011
• Status: Active Grant

• Current Assignee: General Motors LLC (General Motors Company)
• Original Assignee: General Motors LLC (General Motors Company)
• Inventors: Talwar, Gaurav, Zhao, Xufang
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Sirjani, Fariba
• Application Number:
  US13/072,003
• Publication Number:
  US 20120245934A1
• Time in Patent Office: 1,712 Days
• Field of Search: 704231-257
• US Class Current: 1/1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method of automatic speech recognition. An utterance is received from a user in reply to a text message, via a microphone that converts the reply utterance into a speech signal. The speech signal is processed using at least one processor to extract acoustic data from the speech signal. An acoustic model is identified from a plurality of acoustic models to decode the acoustic data, and using a conversational context associated with the text message. The acoustic data is decoded using the identified acoustic model to produce a plurality of hypotheses for the reply utterance.

Claims

1. A method of automatic speech recognition, comprising the steps of:

a) receiving a text message at a speech recognition client device;

b) processing the text message with conversational context-specific language models and emotional context-specific language models stored on the client device using at least one processor of the client device to identify a conversational context and an emotional context corresponding to the text message;

c) synthesizing speech from the text message;

d) communicating the synthesized speech via a loudspeaker of the client device to a user of the client device;

e) receiving a reply utterance in response to the text message from the user via a microphone of the client device that converts the reply utterance into a speech signal;

f) pre-processing the speech signal using the at least one processor to extract acoustic data from the received speech signal;

g) communicating the extracted acoustic data, the identified conversational context, and identified emotional context to a speech recognition server;

h) identifying an acoustic model of a plurality of acoustic models stored at the server to be used for decoding the acoustic data based on the identified conversational context, the identified emotional context, or both;

i) decoding the acoustic data using the identified acoustic model to produce a plurality of hypotheses for the reply utterance; and

j) post-processing the plurality of hypotheses to identify one of the hypotheses as the reply utterance;

k) presenting the identified hypothesis to the user;

l) seeking confirmation from the user that the identified hypothesis is correct;

m) outputting the identified hypothesis as at least part of a reply text message if the user confirms that the identified hypothesis is correct;

otherwise

n) using the emotional context to improve identification of the acoustic model, and repeating steps e) through m).

2. A method of automatic speech recognition, comprising the steps of:

a) receiving a text message at a speech recognition client device;

b) processing the text message with conversational context-specific language models and emotional context-specific language models stored on the client device using at least one processor of the client device to identify a conversational context and emotional context corresponding to the text message;

c) synthesizing speech from the text message;

d) communicating the synthesized speech via a loudspeaker of the client device to a user of the client device;

e) receiving a reply utterance in response to the text message from the user via a microphone of the client device that converts the reply utterance into a speech signal;

f) pre-processing the speech signal using the at least one processor to extract acoustic data from the received speech signal;

g) identifying an acoustic model of a plurality of acoustic models to decode the acoustic data, using the identified conversational context and emotional context associated with the text message;

h) decoding the acoustic data using the identified acoustic model to produce a plurality of hypotheses for the reply utterance;

i) determining whether a confidence value associated with at least one of the plurality of hypotheses for the reply utterance is greater or less than a confidence threshold;

j) communicating the extracted acoustic data, the conversational context, and the emotional context to a speech recognition server, if the confidence value is determined to be less than the confidence threshold, otherwise post-processing the plurality of hypotheses to identify one of the hypotheses as the reply utterance, and outputting from the client device the identified hypothesis as at least part of a reply text message;

k) identifying at the server, an acoustic model of a plurality of acoustic models stored at the server to decode the acoustic data, using the identified conversational context, the emotional context, or both;

l) decoding the acoustic data using the acoustic model identified at the server to produce a plurality of hypotheses for the reply utterance;

m) post-processing the plurality of hypotheses to identify one of the hypotheses as the reply utterance; and

n) outputting from the server the identified hypothesis as at least part of a reply text message.