METHOD FOR OPERATING SPEECH RECOGNITION SERVICE AND ELECTRONIC DEVICE SUPPORTING THE SAME

• US 20180285070A1
• Filed: 03/27/2018
• Published: 10/04/2018
• Est. Priority Date: 03/28/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: YOON, Young Min, OH, Young Il, KANG, Hyun Woo, KIM, Jin Yeol, KIM, Ji Hyun, LEE, Da Som, JEON, Yong Joon
• Granted Patent: US 11,733,964 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

An electronic device is provided. The electronic device includes a processor, and a memory, wherein the memory stores instructions that, when executed, cause the processor to receive a user input including a request for performing a first task that requires at least one parameter for execution and not including the entire at least one parameter, transmit first data related to the user input to an external server, receive a first sequence of states of the electronic device for performing the first task from the external server, perform not all but some of the first sequence of the states while displaying at least some of states changed in the first sequence, and after the performing of the some of the first sequence, display a GUI that is required for performing the first task and represents that a user is requested to provide at least one parameter omitted in the user input.

Claims

1. An electronic device comprising:

a housing;

a display exposed through at least a portion of the housing;

a microphone;

a speaker;

a communication circuit;

a processor electrically connected to the display, the microphone, the speaker, and the communication circuit; and

a memory electrically connected to the processor,

wherein the memory is configured to store at least one application program, and

wherein the processor, upon execution of instructions of the at least one application program, is configured to;

receive a first user input including a request for performing a first task that requires at least one parameter for execution and not including an entirety of the at least one parameter,

transmit first data related to the first user input to an external server through the communication circuit,

receive a first sequence of states of the electronic device for performing the first task from the external server through the communication circuit,

perform not all but some of the first sequence of the states while displaying at least some of states changed in the first sequence through the display, by the electronic device, and

after the performing of the some of the first sequence, display a graphical user interface (GUI) that is required for performing the first task and represents that a user is requested to provide at least one parameter omitted in the first user input in a part of the display.

2. The electronic device of claim 1, wherein the GUI comprises at least one of a letter, a text, or a symbol.

3. The electronic device of claim 1, wherein the processor is further configured to:

before the displaying of the GUI, transmit second data related to a state of the electronic device that is a performance result of the some of the first sequence to the external server through the communication circuit.

4. The electronic device of claim 3, wherein the second data comprises an identifier that represents the state of the electronic device.

5. The electronic device of claim 4, wherein the processor is further configured to:

receive a second user input including the omitted parameter through the microphone or a user interface on the display;

transmit third data related to the second user input to the external server through the communication circuit; and

receive a second sequence of states of the electronic device from the external server through the communication circuit.

6. The electronic device of claim 5, wherein the processor is further configured to:

change the GUI according to the reception of the second user input.

7. The electronic device of claim 6, wherein the instructions cause the processor to:

remove the GUI after the second user input is completely received.

8. The electronic device of claim 5, wherein the processor is further configured to:

determine whether the second sequence of the states comprises some of the first sequence of the states that have been already performed for performing the first task; and

complete the first task by using the second sequence of the states while not repeating the performed some of the first sequence.

9. An electronic device supporting an operation of a speech recognition service, the electronic device comprising:

a memory configured to store at least one application;

a display configured to output an execution screen of the application;

a microphone configured to receive a speech signal;

a communication circuit configured to perform communication with at least one external device; and

a processor electrically connected to the memory, the display, the microphone, and the communication circuit,

wherein the processor is configured to;

in relation to a request for performance of a task of the electronic device, receive a first user input not including at least some of one or more pieces of parameter information that is necessary for performance of the task and transmit the first user input to a first external device,

receive a first sequence including at least one unit operation created based on the first user input in relation to performance of the task of the electronic device from the first external device,

output a result screen by performing a first unit operation included in the first sequence,

at a time point of performance of a second unit operation that is performed based on first parameter information sequentially included in the first sequence after the first unit operation and not included in the first user input, determine whether the first parameter information of the first user input is omitted,

determine an attribute of the first parameter information, and

display a specific graphical user interface (GUI) in an area of the result screen of the first unit operation, which corresponds to an attribute of the first parameter information.

10. The electronic device of claim 9, wherein the processor is further configured to:

after the displaying of the GUI, receive a second user input including at least one of a speech input or a touch input including the first parameter information.

11. The electronic device of claim 10, wherein the processor is further configured to:

endow the GUI with a specific effect at a time point of reception of the second user input.

12. A method for operating a speech recognition service of an electronic device, the method comprising:

in relation to a command on performance of a task of the electronic device, receiving a first user input not including at least some of one or more pieces of parameter information that is necessary for performance of the task;

transmitting the first user input to a first external device;

receiving a first sequence including at least one unit operation created based on the first user input in relation to performance of the task of the electronic device from the first external device;

at a time point of performance a second unit operation that is performed based on first parameter information sequentially included in the first sequence after a first unit operation and not included in the first user input, determining whether the first parameter information of the first user input is omitted;

determining an attribute of the first parameter information; and

displaying a specific graphical user interface (GUI) in an area of a screen of the electronic device, which corresponds to an attribute of the first parameter information.

13. The method of claim 12, wherein the determining of whether the first parameter information of the first user input is omitted comprises:

transmitting at least one piece of information on a performance result of the first unit operation or information on determination of whether the first parameter information is omitted to the first external device.

14. The method of claim 12, further comprising:

after the displaying of the GUI, receiving a second user input including the first parameter information and transmitting the second user input to the first external device.

15. The method of claim 14, wherein the receiving of the second user input and the transmitting of the second user input to the first external device comprises:

endowing the GUI with a specific effect at a time point of reception of the second user input.

16. The method of claim 15, wherein the endowing of the GUI with the specific effect comprises:

extinguishing a first GUI displayed and displaying a second GUI, which is endowed with a specific effect.

17. The method of claim 14, further comprising:

completing performance of a task of the electronic device by receiving a second sequence, on which the second user input is reflected, from the first external device.

18. The method of claim 17, wherein the completing of the performance of the task of the electronic device comprises:

determining whether the first unit operation of the at least one unit operation included in the second sequence, which has been performed in advance, is included.

19. The method of claim 12, wherein the displaying of the specific GUI occurs at a performance completion time point of the first unit operation.

20. The method of claim 12, further comprising:

outputting a result screen corresponding to the performed unit operation of the at least one unit operation included in the first sequence.