Server-provided visual output at a voice interface device

• US 10,339,769 B2
• Filed: 11/16/2017
• Issued: 07/02/2019
• Est. Priority Date: 11/18/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Mixter, Kenneth, Yuan, Yuan
• Primary Examiner(s): Rushing, Mark S
• Application Number:
  US15/815,646
• Publication Number:
  US 20180144590A1
• Time in Patent Office: 593 Days
• Field of Search: 340332, 704275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: G08B 5/36 using visible light sources
• CPC Class Codes: G09G 3/14 Semiconductor devices, e.g. diodes
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2825 Reporting to a device located outside the home and the home network access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; telephonic communication systems adapted for combination with telemetering systems H04M11/002

Abstract

A method at an electronic device with an array of indicator lights includes: obtaining first visual output instructions stored at the electronic device, where the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device; receiving a voice input; obtaining from a remote system a response to the voice input and second visual output instructions, where the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria; executing the response; and displaying visual output on the array of indicator lights in accordance with the second visual output instructions, where otherwise in absence of the second visual output instructions the electronic device displays visual output on the array of indicator lights in accordance with the first visual output instructions.

Claims

1. A method, comprising:

at an electronic device with one or more microphones, a speaker, an array of indicator lights, one or more processors, and memory storing one or more programs for execution by the one or more processors;

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

receiving a voice input;

obtaining from a remote system a response to the voice input and second visual output instructions, wherein the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria;

executing the response;

in response to obtaining the second visual output instructions, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in response to a determination of an absence of the second visual output instructions, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

2. The method of claim 1, wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with executing the response.

3. The method of claim 1, further comprising:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with the response to the voice input.

4. The method of claim 1, wherein the one or more criteria comprise:

the voice input including a request associated with a holiday specified in the voice input.

5. The method of claim 1, wherein the one or more criteria comprise:

a date on which the voice input is received corresponds to a holiday.

6. The method of claim 1, wherein the one or more criteria comprise:

the voice input is associated with an on-going game session at the electronic device.

7. The method of claim 1, wherein the array of indicator lights comprises an array of LED lights.

8. The method of claim 1, wherein the array of indicator lights comprises full-color lights.

9. A method, comprising:

at an electronic device with one or more microphones, a speaker, an array of indicator lights, one or more processors, and memory storing one or more programs for execution by the one or more processors;

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

obtaining from a remote system second visual output instructions;

performing an operation;

in association with performing the operation, determining whether one or more alternate visual output criteria are satisfied;

in accordance with a determination that the alternate visual output criteria are satisfied, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in accordance with the determination that the alternate visual output criteria are not satisfied, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

10. The method of claim 9, wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with performing the operation.

11. The method of claim 9, further comprising:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with performing the operation.

12. The method of claim 9, wherein the one or more alternate visual output criteria comprise:

the operation includes playing a response associated with a holiday.

13. The method of claim 9, wherein the one or more alternate visual output criteria comprise:

the operation is performed on a date that corresponds to a holiday.

14. The method of claim 9, wherein the one or more criteria comprise:

the operation is associated with an on-going game session at the electronic device.

15. The method of claim 9, wherein the electronic device is one of a group of communicatively coupled devices, further comprising, when the operation is directed to the group of devices and is determined by the electronic device to satisfy the alternate visual output criteria, causing each of the group of devices to perform the operation and display light patterns in accordance with the second visual output instructions.

16. An electronic device, comprising:

one or more microphones;

a speaker;

an array of indicator lights;

one or more processors; and

memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for performing;

obtaining first visual output instructions stored at the electronic device, wherein the first visual output instructions control operation of the array of indicator lights based on operating state of the electronic device, including operating states of applications executing thereon;

receiving a voice input;

obtaining from a remote system a response to the voice input and second visual output instructions, wherein the second visual output instructions are provided by the remote system along with the response in accordance with a determination that the voice input satisfies one or more criteria;

executing the response;

in response to obtaining the second visual output instructions, changing operation of the array of indicator lights to respond to the second visual output instructions instead of the first visual output instructions; and

in response to an absence of the second visual output instructions, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

17. The electronic device of claim 16, wherein displaying the visual output on the array of indicator lights in accordance with the second visual output instructions is concurrent with executing the response.

18. The electronic device of claim 17, wherein the instructions further comprise:

obtaining from the remote system audible output instructions associated with the second visual output instructions; and

outputting audible output on the speaker in accordance with the audible output instructions;

wherein the audible output instructions are supplemental to an audible output associated with the response to the voice input.

19. The electronic device of claim 17, wherein the instructions further comprise:

obtaining from a remote system second visual output instructions;

performing an operation;

in association with performing the operation, determining whether one or more alternate visual output criteria are satisfied;

in accordance with a determination that the alternate visual output criteria are satisfied, displaying visual output on the array of indicator lights in accordance with the second visual output instructions; and

in accordance with the determination that the alternate visual output criteria are not satisfied, displaying visual output on the array of indicator lights in accordance with the first visual output instructions.

20. The electronic device of claim 19, wherein the electronic device is one of a group of communicatively coupled devices, further comprising, when the operation is directed to the group of devices and is determined by the electronic device to satisfy the alternate visual output criteria, the instructions further comprise:

causing each of the group of devices to perform the operation and display light patterns in accordance with the second visual output instructions.