Method and device for generating natural language expression by using framework

• US 10,579,726 B2
• Filed: 03/30/2018
• Issued: 03/03/2020
• Est. Priority Date: 04/05/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jae Yong, Um, Tae Kwang, Ku, Han Jun, Park, Sung Pa, Yeo, Jae Yung, Lee, Da Som, Jeon, Yong Joon
• Primary Examiner(s): Nguyen, Khai N.
• Application Number:
  US15/942,241
• Publication Number:
  US 20180293220A1
• Time in Patent Office: 704 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

An electronic device includes a touchscreen display, a wireless communication circuit, a memory storing a framework, and at least one processor. The processor is configured to receive a first event or a second event, to provide the framework with a notification object associated with an event received among the first event and the second event, to parse the notification object to obtain one or more parameters, to select one or more tasks associated with the received event based on at least part of the one or more parameters by using the framework, to select a natural language expression indicating at least one task of the one or more tasks, to provide a user interface including the natural language expression, through the touchscreen display, and to execute the at least one task based at least partly on a user input of the natural language expression provided on the touchscreen display.

Claims

1. An electronic device comprising:

a touchscreen display exposed;

a wireless communication circuit;

a memory; and

at least one processor electrically connected to the touchscreen display, the wireless communication circuit, and the memory,

wherein the at least one processor is configured to;

receive an event among a first event from outside of the electronic device through the wireless communication circuit and a second event generated from inside of the electronic device;

display a notification object associated with the received event on the touchscreen display;

parse the notification object to obtain one or more parameters;

generate one or more tasks associated with the received event based on at least part of the one or more parameters;

generate a natural language expression indicating at least one task of the one or more tasks;

display a user interface including the natural language expression, through the touchscreen display; and

execute the at least one task based at least partly on a user input selecting the natural language expression provided on the touchscreen display.

2. The electronic device of claim 1, wherein the at least one processor is configured to:

receive the at least one task of the one or more tasks from an external electronic device through the wireless communication circuit.

3. The electronic device of claim 2, wherein the at least one processor is configured to:

transmit at least one parameter of the one or more parameters to the external electronic device; and

receive, from the external electronic device, the at least one task as a response to transmitting the at least one parameter.

4. The electronic device of claim 1, wherein the at least one processor is configured to:

receive the natural language expression from an external electronic device by using the wireless communication circuit.

5. The electronic device of claim 4, wherein the at least one processor is configured to:

transmit at least one parameter of the one or more parameters to the external electronic device; and

receive, from the external electronic device, the natural language expression as a response to transmitting the at least one parameter.

6. The electronic device of claim 1, wherein the at least one task includes a first task and a second task, and

wherein the at least one processor is configured to:

display a first natural language expression indicating the first task, and a second natural language expression indicating the second task as at least part of the natural language expression, through the user interface.

7. The electronic device of claim 1, wherein the memory stores a plurality of tasks including the one or more tasks, and

wherein the at least one processor is configured to:

select the one or more tasks from the plurality of tasks.

8. The electronic device of claim 7, wherein the plurality of tasks includes a first set of tasks including one or more first tasks corresponding to a first domain and a second set of tasks including one or more second tasks corresponding to a second domain, and

wherein the at least one processor is configured to:

if the received event corresponds to the first domain, select the one or more first tasks from the first set of tasks; and

if the received event corresponds to the second domain, select the one or more second tasks from the second set of tasks.

9. The electronic device of claim 8, wherein the first domain corresponds to a first application program and the second domain corresponds to a second application program, and

wherein the at least one processor is configured to:

select the one or more first tasks or the one or more second tasks further based on an application program corresponding to the received event.

10. The electronic device of claim 1, wherein the at least one processor is configured to:

obtain context information of the electronic device; and

generate the one or more tasks based at least in part on the context information.

11. The electronic device of claim 10, wherein the context information of the electronic device includes information of an application program being executed by the electronic device.

12. The electronic device of claim 1, wherein the first event from outside of the electronic device includes a reception of a call, a short message, or an instant message, which is associated with a specified application program.

13. The electronic device of claim 1, wherein the second event generated from inside of the electronic device is associated with power management, a memory state, or security of the electronic device.

14. The electronic device of claim 1, wherein the user input corresponds to a touch input to the touchscreen display or a voice input.

15. A method of generating a hint, the hint including a natural language expression, the method comprising:

receiving an event among a first event from outside of an electronic device and a second event generated from inside of the electronic device;

displaying a notification object associated with the received event on a touchscreen display of the electronic device;

parsing the notification object to obtain one or more parameters;

generating one or more tasks associated with the received event based on at least part of the one or more parameters;

generating a request for a natural language expression indicating at least one task of the one or more tasks;

displaying a user interface including the natural language expression through a touchscreen display; and

executing the at least one task based at least partly on a user input selecting the natural language expression through the touchscreen display.

16. The method of claim 15, further comprising:

receiving the at least one task of the one or more tasks from an external electronic device, before the selecting of the one or more tasks.

17. The method of claim 16, further comprising:

transmitting at least one parameter of the one or more parameters to the external electronic device, before the receiving of the at least one task,

wherein the at least one task is received as a response to transmitting the at least one parameter.

18. The method of claim 15, wherein the selecting of the one or more tasks includes selecting the one or more tasks from a plurality of tasks stored in a memory of the electronic device.

19. The method of claim 18, wherein the plurality of tasks includes a first set of tasks including one or more first tasks corresponding to a first domain and a second set of tasks including one or more second tasks corresponding to a second domain, and

wherein the generating of the one or more tasks includes:

if the received event corresponds to the first domain, selecting the one or more first tasks from the first set of tasks; and

if the received event corresponds to the second domain, selecting the one or more second tasks from the second set of tasks.

20. The method of claim 15, further comprising:

before the generating of the one or more tasks, obtaining context information of the electronic device,

wherein the generating of the one or more tasks includes selecting the one or more tasks based at least partly on the context information.

21. The method of claim 15, wherein the user input corresponds to a touch input to the touchscreen display or a voice input.

22. A non-transitory computer readable storage medium including a plurality of instructions that, when executed by a processor of an electronic device, are configured to cause the processor to:

receive an event among a first event from the outside of the electronic device and a second event generated from inside of the electronic device;

display a notification object associated with the received event on a touchscreen display of the electronic device;

parse the notification object to obtain one or more parameters;

generate one or more tasks associated with the received event based on at least part of the one or more parameters;

generate a request for a natural language expression indicating at least one task of the one or more tasks;

display a user interface including the natural language expression through a touchscreen display; and

execute the at least one task based at least partly on a user input selecting the natural language expression through the touchscreen display.

23. The non-transitory computer readable storage medium of claim 22, wherein the user input corresponds to a touch input to the touchscreen display or a voice input.