Method for operating speech recognition service, electronic device and system supporting the same

• US 10,847,152 B2
• Filed: 03/28/2018
• Issued: 11/24/2020
• Est. Priority Date: 03/28/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Oh, Byong Mok, Ku, Han Jun, Woo, Kyoung Gu, Lee, Sang Hoon, Rhee, In Jong, Jung, Seok Yeong, Kim, Kyung Tae, Kim, Ji Hyun, Jang, Dong Ho
• Primary Examiner(s): Jackson, Jakieda R
• Application Number:
  US15/938,549
• Publication Number:
  US 20180286401A1
• Time in Patent Office: 972 Days
• Field of Search: 7042701, 704 9, 704275
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

An electronic device is provided. The electronic device includes a communication module, a microphone receiving a voice input according to user speech, a memory storing information about an operation of the speech recognition service, a display, and a processor electrically connected with the communication module, the microphone, the memory, and the display. The processor is configured to calculate a specified numerical value associated with the operation of the speech recognition service, to transmit information about the numerical value to a first external device processing the voice input, and to transmit a request for a function, which corresponds to the calculated numerical value, of at least one function associated with the speech recognition service stepwisely provided from the first external device depending on a numerical value, to the first external device to refine a function of the speech recognition service supported by the electronic device.

Claims

1. An electronic system, comprising:

a user device including;

a display,

a microphone,

a speaker,

a communication circuit,

a first memory, and

a first processor electrically connected with the display, the microphone, the speaker, the communication circuit, and the first memory; and

a first server including;

a communication interface,

a second memory, and

a second processor electrically connected with the communication interface and the second memory,

wherein the second memory includes an automatic speech recognition (ASR) module and a natural language understanding (NLU) module,

wherein the first memory stores instructions that, when executed by the first processor, causes the first processor to;

execute an intelligence application supporting a speech recognition service,

receive a voice input of a user through the microphone, under the control of the intelligence application, and

transmit data associated with the voice input to the second processor through the communication circuit, under the control of the intelligence application,

wherein the second memory stores instructions that, when executed by the second processor, causes the second processor to;

process the data by using at least one of the ASR module or the NLU module,

calculate at least one numerical value associated with the at least one of the ASR module or the NLU module by using the data, and

provide the calculated at least one numerical value to the first processor, and

wherein the first memory further stores instructions that, when executed by the first processor, causes the first processor to;

determine a use-experience level of the intelligence application based on the at least one numerical value, and

provide the determined use-experience level on a user interface.

2. An electronic device supporting a speech recognition service, the electronic device comprising:

a communication module configured to communicate with at least one external device;

a microphone configured to receive a voice input according to user speech;

a memory configured to store information about an operation of the speech recognition service;

a display configured to output a screen associated with the operation of the speech recognition service; and

a processor electrically connected with the communication module, the microphone, the memory, and the display,

wherein the processor is configured to;

calculate a specified numerical value associated with the operation of the speech recognition service,

transmit information about the calculated specified numerical value to a first external device processing the voice input,

transmit a request for refining at least one function associated with the speech recognition service supported by the electronic device to the first external device, wherein the at least one function associated with the speech recognition service stepwisely provided from the first external device depending on a numerical value, and

refine the at least one function associated with the speech recognition service supported by the electronic device based on a response to the request.

3. The electronic device of claim 2, wherein the processor is further configured to:

assign a point to at least one of an automatic speech recognition (ASR) module or a natural language understanding (NLU) module included in the first external device in association with function execution of the first external device; and

calculate the specified numerical value based on collection of the assigned point.

4. The electronic device of claim 3, wherein the processor is further configured to:

collect at least one voice input information on which speaker-dependent speech recognition is performed by the ASR module;

accumulate and calculate a user speech time corresponding to the collected at least one voice input information; and

assign the point to the ASR module based on an accumulated amount of the user speech time.

5. The electronic device of claim 3, wherein the processor is further configured to:

assign the point to the ASR module based on generation of a speaker-dependent recognition model with respect to wakeup-command speech associated with the operation of the speech recognition service.

6. The electronic device of claim 3, wherein the processor is further configured to:

in association with a speech recognition execution of the ASR module with respect to the voice input, if an error of the speech recognition execution is revised, assign the point to the ASR module based on a speech recognition model update of the ASR module performed in response to the revision.

7. The electronic device of claim 3, wherein the processor is further configured to:

in association with derivation execution of user speech intent of the NLU module with respect to the voice input, if user preference information provided from a user is applied to at least one of a domain, an intent, or a parameter associated with the voice input obtained by the NLU module, assign the point to the NLU module based on application of the user preference information.

8. The electronic device of claim 3, wherein the processor is further configured to:

in association with derivation execution of user speech intent of the NLU module with respect to the voice input, if at least one function response associated with function control of the electronic device or function control of a second external device interacting with the electronic device is set with respect to a specific user speech intent to be derived by the NLU module, assign the point to the NLU module based on the setting of the at least one function response.

9. The electronic device of claim 2, wherein the processor is further configured to:

assign a point to a third external device receiving and storing at least one of information about the electronic device or information about a user of the electronic device from the electronic device; and

calculate the specified numerical value based on the assigned point.

10. The electronic device of claim 9, wherein the processor is further configured to:

receive and output query information about verification or check of the at least one of the information about the electronic device or the information about the user of the electronic device stored in the third external device, from the first external device.

11. The electronic device of claim 10, wherein the processor is further configured to:

if the at least one of the information about the electronic device or the information about the user of the electronic device is verified or checked by a user feedback associated with the query information, assign the point to the third external device based on the verification or the check of the at least one of the information about the electronic device or the information about the user of the electronic device.

12. A speech recognition service operating method of an electronic device, the method comprising:

receiving a voice input according to user speech;

calculating a specified numerical value associated with an operation of the speech recognition service;

transmitting at least one of information about the voice input or information about the calculated specified numerical value to a first external device processing the voice input;

transmitting a request for refining at least one function associated with the speech recognition service stepwisely provided from the first external device depending on the calculated specified numerical value, to the first external device;

receiving a response to the request from the first external device; and

refine the at least one function associated with the speech recognition service based the response.

13. The method of claim 12, wherein the calculating includes:

assigning a point to at least one of an automatic speech recognition (ASR) module or a natural language understanding (NLU) module included in the first external device in association with function execution of the first external device; and

calculating the specified numerical value based on collection of the assigned point.

14. The method of claim 13, wherein the assigning includes:

collecting at least one voice input information on which speaker-dependent speech recognition is performed by the ASR module;

accumulating and calculating a user speech time corresponding to the collected at least one voice input information; and

assigning the point to the ASR module based on an accumulated amount of the user speech time.

15. The method of claim 13, wherein the assigning includes:

assigning the point to the ASR module based on generation of a speaker-dependent recognition model with respect to wakeup-command speech associated with the operation of the speech recognition service.

16. The method of claim 13, wherein the assigning includes:

if an error of a speech recognition execution of the ASR module with respect to the voice input is revised, assigning the point to the ASR module based on a speech recognition model update of the ASR module performed in response to the revision.

17. The method of claim 13, wherein the assigning includes:

if user preference information provided from a user is applied to at least one of a domain, an intent, or a parameter associated with the voice input obtained by the NLU module in an operation of deriving user speech intent of the NLU module with respect to the voice input, assigning the point to the NLU module based on application of the user preference information.

18. The method of claim 13, wherein the assigning includes:

if at least one function response associated with function control of the electronic device or function control of a second external device interacting with the electronic device is set with respect to a specific user speech intent to be derived by the NLU module deriving user speech intent associated with the voice input, assigning the point to the NLU module based on the setting of the at least one function response.

19. The method of claim 12, wherein the calculating includes:

assigning a point to a third external device storing at least one of information about the electronic device or information about a user of the electronic device; and

calculating the specified numerical value based on the assigned point.

20. The method of claim 19, wherein the assigning includes:

receiving and outputting query information about verification or check of the at least one of the information about the electronic device or the information about the user of the electronic device stored in the third external device, from the first external device;

verifying and checking the at least one of the information about the electronic device or the information about the user of the electronic device by a user feedback associated with the query information; and

assigning the point to the third external device based on the verification or the check of the at least one of the information about the electronic device or the information about the user of the electronic device.