DIGITAL ASSISTANT PROVIDING AUTOMATED STATUS REPORT

• US 20170358300A1
• Filed: 09/15/2016
• Published: 12/14/2017
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: LAURENS, Peter Allan, VERWYMEREN, Christopher, BOOKER, Susan L., MOORE, Jonathan J., MALANI, Roshni, PHIPPS, Benjamin S.
• Granted Patent: US 10,490,187 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Systems and processes for operating a digital assistant are provided. In one example process, a speech input is received from a user. A user intent is determined based on the speech input. Determining the user intent includes generating text based on the speech input, performing natural language processing of the text, and determining the user intent based on a result of the natural language processing. In accordance with the user intent, status information associated with at least one of the one or more electronic devices is requested. The status information associated with the at least one of the one or more electronic devices is received. A spoken output is generated and represents the status information associated with the at least one of the one or more electronic devices. The spoken output is caused to be provided to the user.

Claims

1. A user device, the user device being operable to communicate with one or more electronic devices, comprising:

one or more processors;

memory; and

one or more programs stored in memory, the one or more programs including instructions for;

receiving a speech input;

determining a user intent based on the speech input, wherein determining the user intent comprises;

generating text based on the speech input,

performing natural language processing of the text, and

determining the user intent based on a result of the natural language processing;

requesting, in accordance with the user intent, status information associated with at least one of the one or more electronic devices;

receiving the status information associated with the at least one of the one or more electronic devices;

generating a spoken output representing the status information associated with the at least one of the one or more electronic devices; and

causing the spoken output to be provided to the user.

2. The user device of claim 1, wherein the status information associated with the at least one of the one or more electronic devices includes at least one of battery status information, connectivity status information, operating mode status information, location status information, or device configuration status information.

3. The user device of claim 1, wherein receiving a speech input comprises:

receiving the speech input at an invoking device from a user, wherein the invoking device includes the user device or an electronic device that is communicatively connected to the user device.

4. The user device of claim 1, wherein determining the user intent comprises:

determining whether the speech input includes one or more references identifying at least one of the one or more electronic devices;

in accordance with a determination that the speech input includes one or more references identifying at least one of the one or more electronic devices, determining the user intent based on the one or more references; and

in accordance with a determination that the speech input includes information that does not identify at least one of the one or more electronic devices, determining the user intent based on at least one of an expression of the speech input or one or more keywords included in the speech input.

5. The user device of claim 4, wherein the speech input includes the information in addition to the one or more references identifying at least one of the one or more electronic devices.

6. The user device of claim 4, wherein the speech input includes the information without a reference identifying at least one of the one or more electronic devices.

7. The user device of claim 4, wherein determining the user intent based on at least one of an expression of the speech input or one or more keywords included in the speech input comprises:

determining whether the speech input includes a singular form expression or a plural form expression;

in accordance with a determination that the speech input includes a singular form expression, determining, from the one or more electronic devices, a single electronic device for which status information is to be requested; and

in accordance with a determination that the speech input includes a plural form expression, determining, from the one or more electronic devices, at least two electronic devices for which status information is to be requested.

8. The user device of claim 7, wherein the single electronic device for which the status information is to be requested includes an invoking device, wherein the invoking device receives the speech input from the user.

9. The user device of claim 7, wherein the at least two electronic devices for which the status information is to be requested includes an invoking device, wherein the invoking device receives the speech input from the user.

10. The user device of claim 4, wherein determining the user intent based on at least one of an expression of the speech input or one or more keywords included in the speech input comprises:

determining whether the speech input includes one or more keywords; and

in accordance with a determination that the speech input includes one or more keywords, determining, from the one or more electronic devices, at least one electronic device for which status information is to be requested.

11. The user device of claim 10, wherein the at least one electronic device for which the status information is to be requested includes an invoking device, wherein the invoking device receives the speech input from the user.

12. The user device of claim 1, wherein requesting status information associated with the at least one of the one or more electronic devices comprises:

identifying, based on the determination of the user intent, the at least one of the one or more electronic devices for which the status information is to be requested; and

communicating a request for the status information to the at least one of the one or more electronic devices for which the status information is to be requested.

13. The user device of claim 12, wherein identifying the at least one of the one or more electronic devices for which the status information is to be requested is further based on context information,

wherein the context information indicates at least one of detectability of the one or more electronic devices or availability of the one or more electronic devices.

14. The user device of claim 1, wherein the status information includes battery status of a headset communicatively connected to the user device.

15. The user device of claim 14, wherein receiving the status information associated with the at least one of the one or more electronic devices comprises receiving at least one of:

overall battery status of the headset,

battery status of a left earbud and a right earbud of the headset, and

battery status of a case associated with the headset.

16. The user device of claim 1, wherein generating the spoken output representing the status information associated with the at least one of the one or more electronic devices comprises at least one of:

generating the spoken output in an order that corresponds to an order of the one or more references identifying the at least one of the one or more electronic devices in the speech input;

generating the spoken output using a paraphrased response;

generating the spoken output based on a type of an invoking device, wherein the invoking device receives the speech input from the user;

generating the spoken output based on context information; and

generating the spoken output based on the status information associated with the at least one of the one or more electronic devices.

17. The user device of claim 16, wherein the invoking device is a headset and wherein generating the spoken output based on the type of the invoking device comprises generating the an audio representation of at least one of:

overall battery status of the headset,

battery statuses of a left earbud and a right earbud of the headset, and

battery status of a case associated with the headset.

18. The user device of claim 16, wherein generating the spoken output based on context information comprises:

determining whether a total number of the at least one of the one or more electronic devices satisfies a first threshold;

in accordance with a determination that the total number of the at least one of the one or more electronic devices satisfies the first threshold, generating an audio representation having a first format; and

in accordance with a determination that the total number of the at least one of the one or more electronic devices does not satisfy the first threshold, generating an audio representation having a second format.

19. The user device of claim 16, wherein generating the spoken output based on the status information associated with the at least one of the one or more electronic devices comprises:

determining, for each of the at least one of the one or more electronic devices, whether the status information satisfies a second threshold;

in accordance with a determination that the status information satisfies the second threshold, including an audio representation of the status information associated with at least one of the one or more electronic devices; and

in accordance with a determination that the status information does not satisfy the second threshold, performing at least one of;

excluding the audio representation of the status information associated with the at least one of the one or more electronic devices, and

including an audio representation of substitute information in lieu of the status information.

20. The user device of claim 16, wherein generating the spoken output based on status information associated with the at least one of the one or more electronic devices comprises:

generating the spoken output according to battery status information associated with the at least one of the one or more electronic devices.

21. The user device of claim 1, wherein causing the spoken output to be provided to the user comprises:

causing the spoken output to be provided to the user at the user device.

22. The user device of claim 1, wherein causing the spoken output to be provided to the user comprises:

causing the spoken output to be provided to the user at an electronic device communicatively connected to the user device.

23. A non-transitory computer-readable storage medium comprising one or more programs for execution by one or more processors of a user device, the user device being operable to communicate with one or more electronic devices, the one or more programs including instructions which, when executed by the one or more processors, cause the user device to:

receive a speech input;

determine a user intent based on the speech input, wherein determining the user intent comprises;

generating text based on the speech input,

performing natural language processing of the text, and

determining the user intent based on a result of the natural language processing;

request, in accordance with the user intent, status information associated with at least one of the one or more electronic devices;

receive the status information associated with the at least one of the one or more electronic devices;

generate a spoken output representing the status information associated with the at least one of the one or more electronic devices; and

cause the spoken output to be provided to the user.

24. A method for operating a digital assistant, comprising:

at a user device with one or more processors and memory, the user device operable to communicate with one or more electronic devices;

receiving a speech input;

determining a user intent based on the speech input, wherein determining the user intent comprises;

generating text based on the speech input,

performing natural language processing of the text, and

determining the user intent based on a result of the natural language processing;

requesting, in accordance with the user intent, status information associated with at least one of the one or more electronic devices;

receiving the status information associated with the at least one of the one or more electronic devices;

generating a spoken output representing the status information associated with the at least one of the one or more electronic devices; and

causing the spoken output to be provided to the user.