System and device for selecting speech recognition model

• US 10,777,193 B2
• Filed: 06/27/2018
• Issued: 09/15/2020
• Est. Priority Date: 06/27/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jung In, Kwon, Min Seok, Rhee, In Jong, Lee, Chang Heon, Jung, Seok Yeong
• Primary Examiner(s): Le, Thuykhanh
• Application Number:
  US16/019,740
• Publication Number:
  US 20180374476A1
• Time in Patent Office: 811 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A system includes network interface, processor operatively connected to the network interface, and memory operatively connected to the processor. The memory stores a plurality of language models of a first language. The plurality of language models be different from one another. Each of the plurality of language models is provided for speakers whose native language is different from the first language. The memory further stores instructions that cause the processor to receive a user'"'"'s speech data associated with a plurality of sample texts from an external electronic device, to perform automatic speech recognition (ASR) on the speech data using the plurality of language models, to select one language model from the plurality of language models based on results from the performed ASR, and to use the selected one language model as a default language model for speech of the user.

Claims

1. An electronic device comprising:

a touchscreen;

a microphone;

a wireless communication circuit;

a processor operatively connected to the touchscreen display, the microphone, and the wireless communication circuit; and

a memory operatively connected to the processor, wherein the memory stores a plurality of language models of a first language, wherein the plurality of language models are different from one another, and each of the plurality of language models is provided for speakers whose native language is different from the first language, wherein the memory further stores instructions that, when executed, cause the processor to;

display a plurality of sample texts on the touchscreen display;

receive, through the microphone, a plurality of a user'"'"'s speeches each associated with one of the plurality of sample texts;

transmit data of the plurality of the user'"'"'s speeches to an external server through the wireless communication circuit;

receive results from ASR performed on the data of the plurality of the user'"'"'s speeches from the external server through the wireless communication circuit;

select one language model from the plurality of language models based on the received results from the ASR; and

use the selected language model as a default language model for speech of the user, and

wherein the instructions further cause the processor to;

display, on the touchscreen display, an object for skipping a setting of the default language model; and

when a user input for selecting the object is received through the touchscreen display, allow the external server to select a preset language model as the default language model.

2. The electronic device of claim 1, wherein the instructions further cause the processor to:

determine scores for each of the plurality of language models based on the results from the ASR; and

use the scores to select the one language model from the plurality of language models.

3. The electronic device of claim 2, wherein the instructions further cause the processor to:

weigh the scores by applying a weight to each of the scores, wherein the weight is based on at least one of a sample text and a language model corresponding to each of the scores;

determine recognition accuracy for each language model based on the weighted scores; and

use the recognition accuracy to select the one language model from the plurality of language models.

4. The electronic device of claim 1, wherein the instructions further cause the processor to:

receive, through the microphone, data of the user'"'"'s first speech associated with a first sample text in the plurality of sample texts;

perform the ASR on the data of the user'"'"'s first speech using the plurality of language models;

determine accuracy of the ASR based on the first sample text and the user'"'"'s first speech;

when the accuracy is less than a critical value, re-receive the user'"'"'s first speech through the microphone; and

when the accuracy is equal to or greater than the critical value, receive, through the microphone, the user'"'"'s second speech associated with a second sample text in the plurality of sample texts, wherein the second sample text is different from the first sample text.

5. The electronic device of claim 1, wherein the memory stores personalization information of the user, and

wherein the instructions further cause the processor to:

select the one language model from the plurality of language models based on the personalization information and the results from the ASR.

6. The electronic device of claim 5, wherein the instructions further cause the processor to:

select one or more first language models from the plurality of language models based on the personalization information of the user;

select a second language model from the plurality of language models based on the results from the ASR; and

when the second language model is included in the one or more first language models, select the second language model as the default language model.

7. An electronic device comprising:

a touchscreen display;

a microphone;

a wireless communication circuit;

a processor operatively connected to the touchscreen display, the microphone, and the wireless communication circuit; and

a memory operatively connected to the processor, wherein the memory stores a plurality of language models of a first language, wherein the plurality of language models are different from one another, and each of the plurality of language models is provided for speakers whose native language is different from the first language, wherein the memory further stores instructions that, when executed, cause the processor to;

display a plurality of sample texts on the touchscreen display;

receive, through the microphone, a plurality of a user'"'"'s speeches each associated with one of the plurality of sample texts; and

transmit data of the plurality of the user'"'"'s speeches to an external server through the wireless communication circuit, and

wherein transmitting the data of the plurality of the user'"'"'s speeches causes the external server to;

perform ASR on the data of the plurality of the user'"'"'s speeches using the plurality of language models;

select one language model from the plurality of language models based on results from the ASR; and

use the selected one language model as a default language model for speech of the user,

wherein the external server stores the plurality of language models, and

wherein the instructions further cause the processor to;

display, on the touchscreen display, a first object for skipping a setting of the default language model; and

when a user input for selecting the first object is received through the touchscreen display, allow the external server to select a preset language model as the default language model.

8. The electronic device of claim 7, wherein the memory further stores a plurality of text groups respectively corresponding to the plurality of language models,

wherein each of the plurality of text groups includes one or more sample texts, and

wherein the plurality of sample texts is selected from the plurality of text groups.

9. The electronic device of claim 7, wherein the instructions further cause the processor to:

display, on the touchscreen display, a second object for activating reception of the plurality of the user'"'"'s speeches; and

when a user input for selecting the second object is received through the touchscreen display, receive the plurality of the user'"'"'s speeches.

10. The electronic device of claim 7, wherein the instructions further cause the processor to:

receive, through the microphone, the user'"'"'s first speech associated with a first sample text in the plurality of sample texts;

transmit data of the user'"'"'s first speech to an external server through the wireless communication circuit;

when a first response from the external server is received through the wireless communication circuit, re-receive the user'"'"'s first speech through the microphone; and

when a second response different from the first response is received from the external server through the wireless communication circuit, receive, through the microphone, the user'"'"'s second speech associated with a second sample text in the plurality of sample texts, wherein the second sample text is different from the first sample text,

wherein the first response corresponds to accuracy of ASR on the user'"'"'s first speech being less than a critical value, and

wherein the second response corresponds to the accuracy being equal to or greater than the critical value.

11. The electronic device of claim 10, wherein the instructions further cause the processor to:

when the second response is received, display an indicator at a location corresponding to the first sample text.