Speech-responsive portable speaker

• US 10,079,017 B1
• Filed: 10/10/2017
• Issued: 09/18/2018
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Typrin, Marcello, Yum, Steve Hoonsuck, Hagler, Chris Stewart
• Primary Examiner(s): Pullias, Jesse
• Application Number:
  US15/728,727
• Time in Patent Office: 343 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use

Abstract

An electronic device may operate in different modes of operations. In a first mode of operation, the electronic device may receive user speech via a microphone, generate audio data that represents the user speech, and then send the audio data to one or more remote computing device for analysis. In a second mode of operation, the electronic device may receive user speech via the microphone, generate audio data that represents the user speech, and then analyze the audio data using speech processing. In some instances, the electronic device may operate in the first mode of operation and/or the second mode of operation based on whether the electronic device is receiving power from an external source.

Claims

1. An electronic device comprising:

one or more processors; and

one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

determining that the electronic device is operating in a first power state;

based at least in part on determining that the electronic device is operating in the first power state, causing the electronic device to operate in a first mode;

generating, while the electronic device is operating in the first mode, first audio data representing first user speech;

based at least in part on the electronic device operating in the first mode, sending the first audio data to a speech support service configured to perform speech recognition on the first audio data to identify at least a portion of the first user speech;

determining that the electronic device is operating in a second power state;

based at least in part on determining that the electronic device is operating in the second power state, causing the electronic device to operate in a second mode;

generating, while the electronic device is operating in the second mode, second audio data representing second user speech; and

based at least in part on the electronic device operating in the second mode, performing speech recognition on the second audio data to identify at least a portion of the second user speech.

2. The electronic device as recited in claim 1, the operations further comprising:

determining that the electronic device is receiving external power, wherein determining that the electronic device is operating in the first power state is based at least in part on determining that the electronic device receiving the external power; and

determining that the electronic device is no longer receiving the external power, wherein determining that the electronic device is operating in the second power state is based at least in part on determining that the electronic device no longer receiving the external power.

3. The electronic device as recited in claim 1, the operations further comprising:

determining that the electronic device is receiving external power, wherein determining that the electronic device is operating in the second power state is based at least in part on determining that the electronic device receiving the external power; and

determining that the electronic device is no longer receiving the external power, wherein determining that the electronic device is operating in the first power state is based at least in part on determining that the electronic device no longer receiving the external power.

4. The electronic device as recited in claim 1, further comprising at least one battery, and wherein:

determining the electronic device is operating in the first power state is based at least in part on a first charge level associated with the at least one battery being below a threshold; and

determining the electronic device is operating in the second power state is based at least in part on a second charge level associated with the at least one battery exceeding the threshold.

5. The electronic device as recited in claim 1, further comprising at least one speaker, and wherein the operations further comprise, before determining that the electronic device is operating in the second mode:

receiving, third audio data from the speech support service; and

outputting, using the at least one speaker, audible content represented by the third audio data.

6. The electronic device as recited in claim 1, further comprising at least one speaker, and wherein the operations further comprise:

identifying third audio data using the portion of the second user speech; and

outputting, using the at least one speaker, audible content represented by the third audio data.

7. An electronic device comprising:

one or more processors; and

one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

determining that the electronic device is operating in a first power state;

generating, while the electronic device is operating in the first power state, first audio data representing first user speech;

based at least in part on determining that the electronic device is operating in the first power state, sending the first audio data to a speech support service;

determining that the electronic device is operating in a second power state;

generating, while the electronic device is operating in the second power state, second audio data representing second user speech; and

based at least in part on determining that the electronic device is operating in the second power state, analyzing the second audio data.

8. The electronic device as recited in claim 7, the operations further comprising:

causing the electronic device to operate in a first mode based at least in part on determining that the electronic device is operating in the first power state; and

causing the electronic device to operate in a second mode based at least in part on determining that the electronic device is operating in the second power state.

9. The electronic device as recited in claim 7, wherein:

determining that the electronic device is operating in the first power state is based at least in part on the electronic device receiving external power; and

determining that the electronic device is operating in the second power state is based at least in part on the electronic device no longer receiving the external power.

10. The electronic device as recited in claim 7, wherein:

determining that the electronic device is operating in the second power state is based at least in part on the electronic device receiving external power; and

determining that the electronic device is operating in the first power state is based at least in part on the electronic device no longer receiving the external power.

11. The electronic device as recited in claim 7, further comprising at least one battery, and wherein:

determining that the electronic device is operating in the first power state is based at least in part on a first charge level associated with the at least one battery being below a threshold; and

determining that the electronic device is operating in the second power state is based at least in part on a second charge level associated with the at least one battery exceeding the threshold.

12. The electronic device as recited in claim 7, further comprising at least one speaker, and wherein, before determining that the electronic device is operating in the second power state, the operations further comprise:

receiving third audio data from the speech support service; and

outputting, using the at least one speaker, audible content represented by the third audio data.

13. The electronic device as recited in claim 7, further comprising at least one speaker, and wherein the operations further comprise:

identifying, based at least in part on analyzing the second audio data, third audio data that is associated with the second user speech; and

outputting, using the at least one speaker, audible content represented by the third audio data.

14. A method comprising:

determining that an electronic device is operating in a first power state;

generating, while the electronic device is operating in the first power state, first audio data representing first user speech;

based at least in part on determining that the electronic device is operating in the first power state, sending the first audio data to a speech support service configured to perform speech recognition on the first audio data to identify at least a portion of the first user speech;

determining that the electronic device is operating in a second power state;

generating, while the electronic device is operating in the second power state, second audio data representing second user speech; and

based at least in part on determining that the electronic device is operating in the second power state, performing speech recognition on the second audio data to identify at least a portion of the second user speech.

15. The method as recited in claim 14, wherein:

determining that the electronic device is operating in the first power state comprises determining that the electronic device is operating in the first power state based at least in part on the electronic device receiving external power; and

determining that the electronic device is operating in the second power state comprises determining that the electronic device is operating in the second power state based at least in part on the electronic device no longer receiving the external power.

16. The method as recited in claim 14, wherein:

determining that the electronic device is operating in the second power state comprises determining that the electronic device is operating in the second power state based at least in part on the electronic device receiving external power; and

determining that the electronic device is operating in the first power state comprises determining that the electronic device is operating in the first power state based at least in part on the electronic device no longer receiving the external power.

17. The method as recited in claim 14, further comprising:

causing the electronic device to operate in a first mode based at least in part on determining that the electronic device is operating in the first power state; and

causing the electronic device to operate in a second mode is based at least in part on determining that the electronic device is operating in the second power state.

18. The method as recited in claim 14, wherein, before determining that the electronic device is operating in the second power state, the method further comprises:

receiving third audio data from the speech support service; and

outputting, using at least one speaker, audible content represented by the third audio data.

19. The method as recited in claim 14, further comprising:

identifying third audio data using the portion of the second user speech; and

outputting, using at least one speaker, audible content represented by the third audio data.

20. The method as recited in claim 14, further comprising:

determining that a first charge level associated with at least one battery is below a threshold, wherein determining that the electronic device is operating in the first power state is based at least in part on determining that the first charge level being below the threshold; and

determining that a second charge level associated with the at least one battery exceeds the threshold, wherein determining that the electronic device is operating in the second power state is based at least in part on determining that the second charge level exceeding the threshold.