LED design language for visual affordance of voice user interfaces

• US 10,304,450 B2
• Filed: 05/10/2017
• Issued: 05/28/2019
• Est. Priority Date: 05/10/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Tak, Jung Geun, Martin, Amy, McClellan, Willard
• Primary Examiner(s): Bee, Andrew W
• Application Number:
  US15/592,120
• Publication Number:
  US 20170330429A1
• Time in Patent Office: 748 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04N 21/20 Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/40 Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
• CPC Class Codes: H04N 21/4104 Peripherals receiving signals from specially adapted client devices
• CPC Class Codes: H04N 21/4112 having fewer capabilities than the client, e.g. thin client having less processing power or no tuning capabilities
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4751 for defining user accounts, e.g. accounts for children

Abstract

A method is implemented at an electronic device for visually indicating a voice processing state. The electronic device includes at least an array of full color LEDs, one or more microphones and a speaker. The electronic device collects via the one or more microphones audio inputs from an environment in proximity to the electronic device, and processes the audio inputs by identifying and/or responding to voice inputs from a user in the environment. A state of the processing is then determined from among a plurality of predefined voice processing states, and for each of the full color LEDs, a respective predetermined LED illumination specification is determined in association with the determined voice processing state. In accordance with the identified LED illumination specifications of the full color LEDs, the electronic device synchronizes illumination of the array of full color LEDs to provide a visual pattern indicating the determined voice processing state.

Claims

1. A method for visually indicating a voice processing state, comprising:

at an electronic device with an array of full color LEDs, one or more microphones, a speaker, a processor and memory storing at least one program for execution by the processor;

collecting via the one or more microphones audio inputs from an environment in proximity to the electronic device;

processing the audio inputs, wherein the processing includes one or more of identifying and responding to voice inputs from a user in the environment;

determining a state of the processing from among a plurality of predefined voice processing states;

for each of the full color LEDs, identifying a respective predetermined LED illumination specification associated with the determined voice processing state, wherein the illumination specification includes one or more of an LED illumination duration, pulse rate, duty cycle, color sequence and brightness; and

in accordance with the identified LED illumination specifications of the full color LEDs, synchronizing illumination of the array of full color LEDs to provide a visual pattern indicating the determined voice processing state, wherein the visual pattern is displayed on the surface of the electronic device and includes one or more discrete illumination elements, and the full color LEDs are individually illuminated to form each of the one or more discrete illumination elements according to the corresponding predetermined LED illumination specification.

2. The method of claim 1, wherein the plurality of predefined voice processing states include a hot word detection state in which the electronic device is configured to listen to and recognize one or more predefined hot words and does not to send any audio input to a remote server until at least one of the one or more predefined hot words is recognized.

3. The method of claim 1, wherein the electronic device does not include any display screen, and the full color LEDs are hidden under a top surface of the electronic device and invisible to the user when they are not lit, and wherein the top surface of the electronic device is substantially flat and smooth without exposing the full color LEDs.

4. The method of claim 1, wherein the visual pattern includes a start segment, a loop segment and a termination segment, and the loop segment lasts for a length of time associated with the LED illumination durations of the full color LEDs and configured to match a length of the voice processing state.

5. The method of claim 1, wherein in accordance with the determined voice processing state, the colors of the full color LEDs include a predetermined set of colors.

6. The method of claim 5, wherein the predetermined set of colors include Google brand colors including blue, green, yellow and red, and the array of full color LEDs is divided into four quadrants each associated with one of the Google brand colors.

7. The method of claim 1, wherein the array of full color LEDs are physically arranged in a ring.

8. The method of claim 1, wherein:

the electronic device further includes a touch sense array configured to detect touch events on the top surface of the electronic device;

the touch sense array is arranged on a top surface of a circuit board including an array of via holes, and the full color LEDs are disposed within the via holes of the circuit board; and

the circuit board is positioned immediately under the top surface of the electronic device, thereby allowing the full color LEDs and the touch sense array to be disposed immediately under the top surface of the electronic device.

9. The method of claim 1, wherein the visual pattern is consistent with human reactions associated with the voice processing state.

10. The method of claim 1, further comprising:

determining that the voice processing state is associated with one of a plurality of users, wherein identifying the predetermined LED illumination specifications of the full color LEDs includes customizing at least one of the predetermined LED illumination specifications of the full color LEDs according to an identity of the one of the plurality of users.

11. A non-transitory computer-readable medium, having instructions stored thereon, which when executed by one or more processors cause the processors to perform operations comprising:

at an electronic device with an array of full color LEDs, one or more microphones, and a speaker;

collecting via the one or more microphones audio inputs from an environment in proximity to the electronic device;

processing the audio inputs, wherein the processing includes one or more of identifying and responding to voice inputs from a user in the environment;

determining a state of the processing from among a plurality of predefined voice processing states;

for each of the full color LEDs, identifying a respective predetermined LED illumination specification associated with the determined voice processing state, wherein the illumination specification includes one or more of an LED illumination duration, pulse rate, duty cycle, color sequence and brightness; and

in accordance with the identified LED illumination specifications of the full color LEDs, synchronizing illumination of the array of full color LEDs to provide a visual pattern indicating the determined voice processing state, wherein the visual pattern is displayed on the surface of the electronic device and includes one or more discrete illumination elements, and the full color LEDs are individually illuminated to form each of the one or more discrete illumination elements according to the corresponding predetermined LED illumination specification.

12. The non-transitory computer-readable medium of claim 11, wherein in accordance with a determination that the determined voice processing state is a hot word detection state that occurs when one or more predefined hot words are detected, the array of full color LEDs is divided into a plurality of diode groups that are alternately arranged and configured to be lit sequentially, and diodes in each of the plurality of diode groups are lit with different colors.

13. The non-transitory computer-readable medium of claim 11, wherein in accordance with a determination that the determined voice processing state is a listening state that occurs when the electronic device is actively receiving the voice inputs from the environment and providing received voice inputs to a remote server, all full color LEDs are lit up with a single color, and each full color LED illuminates with different and varying brightness.

14. The non-transitory computer-readable medium of claim 11, wherein the electronic device does not include any display screen, and the full color LEDs are hidden under a top surface of the electronic device and invisible to the user when they are not lit.

15. An electronic device, comprising:

an array of full color LEDs;

one or more microphones;

a speaker;

one or more processors; and

memory having instructions stored thereon, which when executed by the one or more processors cause the processors to perform operations for;

collecting via the one or more microphones audio inputs from an environment in proximity to the electronic device;

processing the audio inputs, wherein the processing includes one or more of identifying and responding to voice inputs from a user in the environment;

determining a state of the processing from among a plurality of predefined voice processing states;

for each of the full color LEDs, identifying a respective predetermined LED illumination specification associated with the determined voice processing state, wherein the illumination specification includes one or more of an LED illumination duration, pulse rate, duty cycle, color sequence and brightness; and

in accordance with the identified LED illumination specifications of the full color LEDs, synchronizing illumination of the array of full color LEDs to provide a visual pattern indicating the determined voice processing state, wherein the visual pattern is displayed on the surface of the electronic device and includes one or more discrete illumination elements, and the full color LEDs are individually illuminated to form each of the one or more discrete illumination elements according to the corresponding predetermined LED illumination specification.

16. The electronic device of claim 15, wherein in accordance with a determination that the voice processing state is a thinking state that occurs when the electronic device is processing the voice inputs received from the user, an increasing number of full color LEDs are lit up during a first illumination cycle of the LED illumination duration, and a decreasing number of full color LEDs are lit up during a second illumination cycle following the first illumination cycle.

17. The electronic device of claim 15, wherein in accordance with a determination that the voice processing state is a responding state that occurs when the electronic device broadcasts a voice message in response to the voice inputs received from the user, a subset of the full color LEDs are lit up with a single color of distinct and varying brightness, and variation of the brightness of each of the subset of the fully color LEDs is consistent with a voice speed associated with the voice inputs from the user.

18. The electronic device of claim 15, wherein the plurality of predefined voice processing states includes one or more of a hot word detection state, a listening state, a thinking state and a responding state.

19. The electronic device of claim 15, wherein the visual pattern includes a start segment, a loop segment and a termination segment, and the loop segment lasts for a length of time associated with the LED illumination durations of the full color LEDs and configured to match a length of the voice processing state.

20. The electronic device of claim 15, wherein the electronic device further includes a touch sense array configured to detect touch events on the top surface of the electronic device.