SPEECH PROCESSING SYSTEM AND TERMINAL

• US 20170148436A1
• Filed: 07/01/2013
• Published: 05/25/2017
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: National Institute of Information and Communications Technology
• Original Assignee: National Institute of Information and Communications Technology
• Inventors: SUGIURA, Komei, OKUMA, Hideo, KIMURA, Noriyuki, SHIGA, Yoshinori, HAYASHI, Teruaki, MIZUKAMI, Etsuo
• Granted Patent: US 9,824,687 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 25/69 for evaluating synthetic or decoded voice signals

Abstract

[Object] An object is to provide an easy-to-use speech processing system attaining higher accuracy of speech recognition.

[Solution] Receiving a speech utterance, the speech processing system performs speech recognition and displays a text 158 of the recognition result. Further, the speech processing system translates the recognition result in accordance with settings to a text 176 of another language and displays and synthesizes speech of the translated result. Further, the speech processing system selects utterance candidates having high possibility to be uttered as the next utterance and having high translation and speech recognitions scores, using outputs of various sensors at the time of utterance, a pre-trained utterance sequence model and translation and speech recognition scores of utterance candidates, and recommends utterance candidates in the form of an utterance candidate recommendation list 192. A user can think of what to say next using the utterances in utterance candidate recommendation list 192 as hints.

Claims

1. A speech processing system, comprising:

utterance input means for receiving an input of utterance information including a speech signal representing an utterance and prescribed environmental information representing an environment in which the utterance is made;

speech recognition means for performing speech recognition on the speech signal in the utterance information received by said utterance input means and for outputting a recognition result as a text;

data processing means for executing a prescribed data processing on the text output by said speech recognition means;

utterance sequence model storage means for storing an utterance sequence model statistically trained such that upon reception of a text of an utterance and said prescribed environmental information, a probability of an utterance in a prescribed set of utterances to be uttered successively following the utterance represented by said text can be calculated;

utterance storage means for storing utterances in said prescribed set of utterances and degree of confidence of data processing when each of said utterances in said set of utterances is processed by said data processing means; and

utterance candidate recommendation means, for scoring, in said set of utterances, candidates of utterances to be recommended to a user who made the utterance recognized by said speech recognition means, based on an evaluation score obtained by combining, in a prescribed form, a probability calculated for each utterance in said prescribed set by said utterance sequence model stored in said utterance sequence model storage means, using the result of recognition by said speech recognition means of the utterance information received by said utterance input means and the environmental information included in the speech information, and the degree of confidence of said data processing on each utterance in said prescribed set of utterances, and for recommending an utterance candidate to the user based on the scores.

2. The speech processing system according to claim 1, wherein

said data processing means includes automatic translation means receiving a result of recognition output from said speech recognition means of a given utterance, for automatically translating the result of recognition to a language different from the language of said given utterance and for outputting the translated result as a text;

and

said degree of confidence is likelihood of the translated result by said automatic translation means being a translation of said given utterance in said different language.

3. The speech processing system according to claim 2, wherein said data processing means further includes speech synthesizing means for synthesizing, based on the text in said different language output from said automatic translation means, a speech signal of said different language.

4. The speech processing system according to claim 1, wherein

said utterance candidate recommendation means includes means for estimating, in said set of utterances, a candidate of an utterance that successively follows the utterance speech-recognized by said speech recognition means, based on an evaluation in terms of a linear sum of the probability calculated by said utterance sequence model for each utterance in said prescribed set and the degree of confidence of each utterance in said prescribed set stored in said utterance storage means;

and

in said linear sum, coefficients of said degree of confidence and said probability are both positive.

5. The speech processing system according to claim 1, further comprising utterance candidate presenting means for presenting to the user an utterance candidate recommended by said utterance candidate recommendation means.

6. The speech processing system according to claim 1, further comprising utterance text information input means, receiving utterance text information including a text representing an utterance and said prescribed environmental information, for applying the text in said utterance text information to said utterance candidate recommendation means and said data processing means, in place of the output of said speech recognition means.

7. A terminal, comprising:

a microphone;

a set of sensors for collecting pieces of information related to surrounding environment;

a display device;

a communication device; and

utterance information transmitting means, connected to said microphone, said set of sensors and said communication device, for transmitting utterance information containing a speech signal obtained from a signal output by said microphone upon reception of an utterance and pieces of information obtained from said set of sensors when said speech signal is obtained, to a prescribed speech processing server through said communication device, and for requesting speech recognition and a prescribed data processing on a result of recognition;

further comprising;

process result presenting means, connected to said communication device, for receiving a process result of said data processing transmitted from said speech processing server in response to said request, and for presenting the process result to a user; and

utterance candidate recommendation list display means, receiving an utterance candidate recommendation list recommended as a plurality of utterance candidates from said speech processing server and displaying the list on said display device, and thereby for recommending utterance candidates to said user.

8. The terminal according to claim 7, wherein

said prescribed data processing performed by said speech processing server on the result of said speech recognition is a process of automatically translating said utterance to a language different from the language of said utterance and further synthesizing a speech of a result of the automatic translation;

the process result of said data processing transmitted from said speech processing server is a speech signal representing the speech synthesized by said speech processing server; and

said process result presenting means includes

a speaker, and

means for driving said speaker with the speech signal representing the speech synthesized by said speech processing server.

9. The terminal according to claim 7, further comprising:

selecting means operable by a user for selecting any of the utterance candidates displayed by said utterance candidate recommendation list; and

utterance text information transmitting means, responsive to selection of any of the utterance candidates in said utterance candidate recommendation list by said selecting means, for transmitting utterance text information including a text of the selected utterance candidate and pieces of information obtained from said set of sensors to a prescribed speech processing server through said communication device, and requesting said prescribed data processing on said utterance text information.