INTELLIGENT DIGITAL ASSISTANT IN A MULTI-TASKING ENVIRONMENT

• US 20180308485A1
• Filed: 06/29/2018
• Published: 10/25/2018
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: KUDURSHIAN, Aram D., JONES, Bronwyn, CRANFILL, Elizabeth Caroline, SADDLER, Harry J.
• Granted Patent: US 10,733,993 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Systems and processes for operating a digital assistant are provided. In one example, a method includes receiving a first speech input from a user. The method further includes identifying context information and determining a user intent based on the first speech input and the context information. The method further includes determining whether the user intent is to perform a task using a searching process or an object managing process. The searching process is configured to search data, and the object managing process is configured to manage objects. The method further includes, in accordance with a determination the user intent is to perform the task using the searching process, performing the task using the searching process; and in accordance with the determination that the user intent is to perform the task using the object managing process, performing the task using the object managing process.

Claims

1-26. -26. (canceled).

27. A method for providing a digital assistant service, comprising:

at a user device with one or more processors and memory;

receiving a speech input from a user to manage one or more system configurations of the user device, wherein the user device is concurrently displaying a plurality of graphical user interfaces associated with a plurality of corresponding applications;

identifying context information associated with the user device;

determining a user intent based on the speech input and context information;

determining whether the user intent indicates an informational request or a request for performing a task related to one of the plurality of corresponding applications;

in accordance with a determination that the user intent indicates an informational request, providing a spoken response to the informational request; and

in accordance with a determination that the user intent indicates a request for performing a task, instantiating a process associated with the user device to perform the task.

28. The method of claim 27, further comprising, prior to receiving the speech input:

displaying, on a display of the user device, an affordance to invoke the digital assistant.

29. The method of claim 28, further comprising:

instantiating the digital assistant service in response to receiving a pre-determined phrase.

30. The method of claim 28, further comprising:

instantiating the digital assistant service in response to receiving a selection of the affordance.

31. The method of claim 27, wherein the one or more system configurations of the user device comprise audio configurations.

32. The method of claim 27, wherein the one or more system configurations of the user device comprise date and time configurations.

33. The method of claim 27, wherein the one or more system configurations of the user device comprise dictation configurations.

34. The method of claim 27, wherein the one or more system configurations of the user device comprise display configurations.

35. The method of claim 27, wherein the one or more system configurations of the user device comprise input device configurations.

36. The method of claim 27, wherein the one or more system configurations of the user device comprise network configurations.

37. The method of claim 27, wherein the one or more system configurations of the user device comprise notification configurations.

38. The method of claim 27, wherein the one or more system configurations of the user device comprise printer configurations.

39. The method of claim 27, wherein the one or more system configurations of the user device comprise security configurations.

40. The method of claim 27, wherein the one or more system configurations of the user device comprise backup configurations.

41. The method of claim 27, wherein the one or more system configurations of the user device comprise application configurations.

42. The method of claim 27, wherein the one or more system configurations of the user device comprise user interface configurations.

43. The method of claim 27, wherein determining the user intent comprises:

determining one or more actionable intents; and

determining one or more parameters associated with the actionable intent.

44. The method of claim 27, wherein the context information comprises at least one of:

user-specific data, device configuration data, and sensor data.

45. The method of claim 27, wherein determining whether the user intent indicates an informational request or a request for performing a task comprises:

determining whether the user intent is to vary a system configuration.

46. The method of claim 27, in accordance with a determination that the user intent indicates an informational request, wherein providing the spoken response to the informational request comprises:

obtaining status of one or more system configurations according to the informational request; and

providing the spoken response according to the status of one or more system configurations.

47. The method of claim 27, in accordance with a determination that the user intent indicates an informational request, further comprising, in addition to providing the spoken response to the informational request:

displaying a first user interface providing information according to the status of the one or more system configurations.

48. The method of claim 27, in accordance with a determination that the user intent indicates an informational request, further comprising, in addition to providing the spoken response to the informational request:

providing a link associated with the informational request.

49. The method of claim 27, in accordance with a determination that the user intent indicates a request for performing a task, instantiating the process associated with the user device to perform the task comprises:

performing the task using the process.

50. The method of claim 49, further comprising:

providing a first spoken output according to a result of performing the task.

51. The method of claim 49, further comprising:

providing a second user interface enabling the user to manipulate a result of performing the task.

52. The method of claim 51, wherein the second user interface comprises a link associated with the result of performing the task.

53. The method of claim 27, in accordance with a determination that the user intent indicates a request for performing a task, instantiating the process associated with the user device to perform the task comprises:

providing a third user interface enabling the user to perform the task.

54. The method of claim 53, wherein the third user interface includes a link enabling the user to perform the task.

55. The method of claim 53, further comprising providing a second spoken output associated with the third user interface.

56. An electronic device comprising:

one or more processors;

memory; and

one or more programs stored in memory, the one or more programs including instructions for performing;

receiving a speech input from a user to manage one or more system configurations of the user device, wherein the user device is concurrently displaying a plurality of graphical user interfaces associated with a plurality of corresponding applications;

identifying context information associated with the user device;

determining a user intent based on the speech input and context information;

determining whether the user intent indicates an informational request or a request for performing a task related to one of the plurality of corresponding applications;

in accordance with a determination that the user intent indicates an informational request, providing a spoken response to the informational request; and

in accordance with a determination that the user intent indicates a request for performing a task, instantiating a process associated with the user device to perform the task.

57. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions which, when executed by the one or more processors, cause the electronic device to perform:

receiving a speech input from a user to manage one or more system configurations of the user device, wherein the user device is concurrently displaying a plurality of graphical user interfaces associated with a plurality of corresponding applications;

identifying context information associated with the user device;

determining a user intent based on the speech input and context information;

determining whether the user intent indicates an informational request or a request for performing a task related to one of the plurality of corresponding applications;

in accordance with a determination that the user intent indicates an informational request, providing a spoken response to the informational request; and

in accordance with a determination that the user intent indicates a request for performing a task, instantiating a process associated with the user device to perform the task.