FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES

• US 20180336905A1
• Filed: 08/16/2017
• Published: 11/22/2018
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: KIM, Yoon, SRISUWANANUKORN, Charles, CARSON, David A., GRUBER, Thomas R., BINDER, Justin G.
• Granted Patent: US 11,217,255 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Systems and processes for operating an intelligent automated assistant to provide extension of digital assistant services are provided. An example method includes, at an electronic device having one or more processors, receiving, from a first user, a first speech input representing a user request. The method further includes obtaining an identity of the first user; and in accordance with the user identity, providing a representation of the user request to at least one of a second electronic device or a third electronic device. The method further includes receiving, based on a determination of whether the second electronic device or the third electronic device, or both, is to provide the response to the first electronic device, the response to the user request from the second electronic device or the third electronic device. The method further includes providing a representation of the response to the first user.

Claims

1. A first electronic device, comprising:

one or more processors;

memory; and

one or more programs stored in memory, the one or more programs including instructions for;

receiving, from a first user, a first speech input representing a user request;

obtaining an identity of the first user;

in accordance with the user identity, providing a representation of the user request to at least one of a second electronic device or a third electronic device;

receiving, based on a determination of whether the second electronic device or the third electronic device, or both, is to provide a response to the first electronic device, the response to the user request from the second electronic device or the third electronic device; and

providing a representation of the response to the first user.

2. The first electronic device of claim 1, wherein the second electronic device is a server remotely disposed from the first electronic device;

and the third electronic device is a client device disposed in the vicinity of the first electronic device.

3. The first electronic device of claim 1, wherein the user request comprises one or more of:

a request for information specific to the first user;

a request for non-user-specific information; and

a request for performing a task.

4. The first electronic device of claim 1, wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

determining whether the third electronic device is communicatively coupled to the first electronic device;

in accordance with a determination that the third electronic device is communicatively coupled to the first electronic device, providing the representation of the user request to the third electronic device and not to the second electronic device; and

in accordance with a determination that the third electronic device is not communicatively coupled to the first electronic device, providing the representation of the user request to the second electronic device.

5. The first electronic device of claim 1, wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

providing the representation of the user request to the second electronic device and not to the third electronic device.

6. The first electronic device of claim 1, wherein providing the representation of the user request to at least one of the second electronic device or the third electronic device comprises:

providing the representation of the user request to both the second electronic device and the third electronic device.

7. The first electronic device of claim 1, wherein the representation of the user request is provided to the third electronic device and not to the second electronic device;

and wherein receiving the response to the user request comprises;

causing the third electronic device to determine whether the third electronic device is capable of providing the response to the user request; and

in accordance with a determination that the third electronic device is capable of providing the response to the user request, receiving, at the first electronic device, the response to the user request from the third electronic device.

8. The first electronic device of claim 7, wherein the one or more programs further comprise instructions for:

in accordance with a determination that the third electronic device is incapable of providing the response to the user request,

forwarding, by the third electronic device, the representation of the user request to the second electronic device; and

receiving, at the first electronic device, the response to the user request from the second electronic device.

9. The first electronic device of claim 1, wherein the representation of the user request is provided to the second electronic device and not to the third electronic device, and wherein receiving the response to the user request comprises:

causing the second electronic device to determine whether the second electronic device is capable of providing the response to the user request; and

in accordance with a determination that the second electronic device is capable of providing the response to the user request, receiving, at the first electronic device, the response to the user request from the second electronic device.

10. The first electronic device of claim 9, wherein the one or more programs further comprise instructions for:

in accordance with a determination that the second electronic device is incapable of providing the response to the user request,

forwarding, by the second electronic device, the representation of the user request to the third electronic device; and

receiving, at the first electronic device, the response to the user request.

11. The first electronic device of claim 10, wherein the first electronic device receives the response to the user request from the third electronic device.

12. The first electronic device of claim 10, wherein the first electronic device receives the response to the user request from the second electronic device based on a response provided by the third electronic device to the second electronic device.

13. The first electronic device of claim 1, wherein the representation of the user request is provided from the first electronic device to both the second electronic device and the third electronic device, and wherein receiving the response to the user request comprises at least one of:

causing the second electronic device to determine whether the second electronic device is capable of providing the response to the user request; and

causing the third electronic device to determine whether the third electronic device is capable of providing the response to the user request.

14. The first electronic device of claim 13, wherein the one or more programs further comprise instructions for:

in accordance with a determination that the second electronic device is capable of providing the response to the user request, and that the third electronic device is incapable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the second electronic device.

15. The first electronic device of claim 13, wherein the one or more programs further comprise instructions for:

in accordance with a determination that the third electronic device is capable of providing the response to the user request, and that the second electronic device is incapable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the third electronic device.

16. The first electronic device of claim 13, wherein the one or more programs further comprise instructions for:

in accordance with a determination that both the second electronic device and the third electronic device are capable of providing the response to the user request,

receiving, at the first electronic device, the response to the user request from the second electronic device or the third electronic device based on a predetermined condition.

17. The first electronic device of claim 1, wherein providing the representation of the response to the first user comprises:

providing, at the first electronic device, a speech output including information in response to the user request.

18. The first electronic device of claim 1, wherein providing the representation of the response to the first user comprises:

providing, at the first electronic device, a speech output associated with performing a task in accordance with the user request.

19. The first electronic device of claim 18, wherein the task is performed by:

the third electronic device;

the first electronic device and the third electronic device;

or

the first electronic device, the third electronic device, and one or more additional electronic devices.

20. The first electronic device of claim 1, wherein the one or more programs further comprise instructions for, prior to providing the representation of the response to the first user:

determining whether the representation of the response is to be provided by the first electronic device;

in accordance with a determination that the representation of the response is to be provided by the first electronic device, providing the representation of the response to the first user by the first electronic device; and

in accordance with a determination that the representation of the response is not to be provided by the first electronic device, forwarding the response to one or more additional electronic devices.

21. The first electronic device of claim 20, wherein determining whether the response is to be provided by the first electronic device is based on at least one of:

the user request; and

at least one of detecting a location of the user or tracking the user'"'"'s movement.

22. The first electronic device of claim 1, wherein the one or more programs further comprise instructions for, prior to providing the representation of the user request to at least one of the second electronic device or the third electronic device:

establishing a connection between the first electronic device and at least one of the second electronic device or the third electronic device.

23. The first electronic device of claim 22, wherein establishing the connection is based on detecting of the third electronic device being within a predetermined distance from the first electronic device.

24. The first electronic device of claim 22, wherein the one or more programs further comprise instructions for:

establishing a connection between the first electronic device and the third electronic device, wherein the third electronic device is associated with the first user; and

establishing a connection between the first electronic device and a fourth electronic device, wherein the fourth electronic device is associated with a second user.

25. The first electronic device of claim 1, wherein the one or more programs further comprise instructions for, establishing one or more connections between the first electronic device and one or more additional electronic devices, wherein the additional electronic devices are the same type of devices as the first electronic device.

26. The first electronic device of claim 1, wherein the one or more programs further comprise instructions for, while providing the response to the first user by the first electronic device,

determining whether the response is to be continually provided at a different electronic device;

and

in accordance with a determination that the response is to be continually provided at a different electronic device, causing the response to be continually provided by at least one of the third electronic device or one or more additional electronic devices.

27. The first electronic device of claim 26, wherein the determination of whether the response is to be continually provided at a different electronic device is based on detecting whether the first user'"'"'s location variation with respect to the first electronic device satisfies a predetermined condition.

28. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:

receive, from a first user, a first speech input representing a user request;

obtain an identity of the first user;

in accordance with the user identity, provide a representation of the user request to at least one of a second electronic device or a third electronic device;

receive, based on a determination of whether the second electronic device or the third electronic device, or both, is to provide a response to the first electronic device, the response to the user request from the second electronic device or the third electronic device; and

provide a representation of the response to the first user.

29. A method for providing a digital assistant service, comprising:

at a first electronic device with one or more processors and memory;

receiving, from a first user, a first speech input representing a user request;

obtaining an identity of the first user;

in accordance with the user identity, providing a representation of the user request to at least one of a second electronic device or a third electronic device;

receiving, based on a determination of whether the second electronic device or the third electronic device, or both, is to provide a response to the first electronic device, the response to the user request from the second electronic device or the third electronic device; and

providing a representation of the response to the first user.