Low power integrated circuit to analyze a digitized audio stream

• US 10,381,007 B2
• Filed: 01/06/2017
• Issued: 08/13/2019
• Est. Priority Date: 12/07/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Liu, Eric, Marti, Stefan Johannes Walter, Kim, Seung Wook
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/400,817
• Publication Number:
  US 20170116992A1
• Time in Patent Office: 949 Days
• Field of Search: 704275
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3293 by switching to a less power-consuming processor, e.g. sub-CPU
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

Methods, devices, and systems for processing audio information are disclosed. An exemplary method includes receiving an audio stream. The audio stream may be monitored by a low power integrated circuit. The audio stream may be digitized by the low power integrated circuit. The digitized audio stream may be stored in a memory, wherein storing the digitized audio stream comprises replacing a prior digitized audio stream stored in the memory with the digitized audio stream. The low power integrated circuit may analyze the stored digitized audio stream for recognition of a keyword. The low power integrated circuit may induce a processor to enter an increased power usage state upon recognition of the keyword within the stored digitized audio stream. The stored digitized audio stream may be transmitted to a server for processing. A response received from the server based on the processed audio stream may be rendered.

Claims

1. A method for processing audio information, comprising:

receiving, by a computing device, an audio stream, the audio stream including a keyword and additional audio;

monitoring, by a low power integrated circuit of the computing device, the audio stream;

digitizing, by the low power integrated circuit, the audio stream, the digitized audio stream including the keyword and the additional audio;

storing the digitized audio stream in a memory in the computing device, wherein storing the digitized audio stream comprises replacing a prior digitized audio stream stored in the memory with the digitized audio stream;

analyzing, by the low power integrated circuit, the stored digitized audio stream for recognition of the keyword;

upon recognition of the keyword within the stored digitized audio stream, inducing, by the low power integrated circuit, a processor of the computing device to enter an increased power usage state and to obtain at least the additional audio of the stored digitized audio stream from the memory;

obtaining, by the processor, at least the additional audio of the stored digitized audio stream from the memory;

transmitting, by the computing device, at least the additional audio of the stored digitized audio stream to a server for determining one or more actions based on at least the additional audio of the stored digitized audio stream; and

rendering, by the computing device, a response received from the server, wherein the response is based on analysis of at least the additional audio of the digitized audio stream and comprises one or more instructions to perform the one or more actions.

2. The method of claim 1, wherein the stored digitized audio stream is converted to a text stream.

3. The method of claim 1, wherein the one or more instructions comprise at least one of performing a web search, dialing a phone number, opening an application, recording text, streaming media, composing a text message, listing directions, or speaking directions.

4. The method of claim 1, wherein the audio stream includes at least one of speech from a user, speech from another computing device, or audio from the another computing device.

5. The method of claim 1, further comprising compressing the digitized audio stream prior to storage in the memory.

6. The method of claim 1, further comprising in response to recognizing the keyword within the stored digitized audio stream:

continuously receiving the audio stream;

continuously monitoring the audio stream; and

continuously digitizing the audio stream.

7. The method of claim 1, further comprising outputting the rendered response by at least one of a display of the computing device or a speaker of the computing device.

8. The method of claim 1, wherein receiving the audio stream comprises receiving the audio stream from a user by a microphone of the computing device.

9. A computing device for processing audio information, comprising:

a low power integrated circuit;

a memory coupled to the low power integrated circuit;

a processor coupled to the low power integrated circuit and the memory;

wherein the low power integrated circuit is configured to;

monitor and digitize an audio stream received by the computing device, the digitized audio stream including a keyword and additional audio;

store the digitized audio stream in the memory, wherein storing the digitized audio stream comprises replacing a prior digitized audio stream stored in the memory with the digitized audio stream;

analyze the stored digitized audio stream to recognize the keyword;

upon recognition of the keyword in the stored digitized audio stream, induce the processor of the computing device to enter an increased power usage state and to obtain at least the additional audio of the stored digitized audio stream from the memory; and

wherein the processor is configured to;

obtain at least the additional audio of the stored digitized audio stream from the memory;

transmit at least the additional audio of the stored digitized audio stream to a server for determining one or more actions based on at least the additional audio of the stored digitized audio stream; and

render a response received by the computing device from a server, wherein the response is based on analysis of at least the additional audio of the digitized audio stream.

10. The computing device of claim 9, wherein the stored digitized audio stream is converted to a text stream.

11. The computing device of claim 9, wherein the response received from the server comprises one or more instructions to perform the one or more actions by the processor.

12. The computing device of claim 11, wherein the one or more instructions comprise at least one of performing, by the processor, a web search, dialing a phone number, opening an application, recording text, streaming media, composing a text message, listing directions, or speaking directions.

13. The computing device of claim 9, wherein the audio stream includes at least one of speech from a user, speech from another computing device, or audio from the another computing device.

14. The computing device of claim 9, wherein the digitized audio stream is compressed prior to storage in the memory.

15. The computing device of claim 9, wherein the low power integrated circuit is further configured to, in response to recognizing the keyword within the stored digitized audio stream:

continuously receive the audio stream;

continuously monitor the audio stream; and

continuously digitize the audio stream.

16. The computing device of claim 9, further comprising at least one of:

a display configured to output the rendered response;

or

a speaker configured to output the rendered response.

17. The computing device of claim 9, wherein the received audio stream is received by a microphone of the computing device.

18. The computing device of claim 9, wherein the processor is further configured to:

transmit the keyword and the additional audio of the stored digitized audio stream to the server for determining the one or more actions.

19. A system for analyzing a digitized audio stream, comprising:

a computing device, comprising;

one or more processors;

a memory; and

a low power integrated circuit coupled to the one or more processors and the memory, the low power integrated circuit configured to;

monitor and digitize an audio stream received by the computing device, the digitized audio stream including a keyword and additional audio;

store the digitized audio stream in the memory;

analyze the stored digitized audio stream to recognize the keyword;

upon recognition of the keyword in the stored digitized audio stream, induce the one or more processors to enter an increased power usage state; and

a server configured to;

receive at least the additional audio of the digitized audio stream transmitted from the computing device;

process at least the additional audio of the received digitized audio stream to determine one or more actions; and

transmit an instruction to perform the one or more actions to the computing device, wherein the instruction is based on analysis of at least the additional audio of the received digitized audio stream.

20. The system of claim 19, wherein the one or more processors are further configured to perform the one or more actions based at least in part on the instruction received from the server.

21. The system of claim 20, wherein the one or more actions comprise at least one of performing a web search, dialing a phone number, opening an application, recording text, streaming media, composing a text message, listing directions, or speaking directions.

22. The system of claim 19, wherein the one or more processors are further configured to transmit at least the additional audio of the digitized audio stream to the server.

23. The system of claim 22, wherein the one or more processors are further configured to compress the digitized audio stream.

24. The system of claim 19, wherein the received audio stream is received by a microphone of the computing device.

25. The system of claim 19, wherein the lower power integrated circuit is further configured to replace a prior digitized audio stream stored in the memory with the digitized audio stream.

26. The system of claim 19, wherein the computing device further comprises at least one of:

a display configured to output the rendered response;

or

a speaker configured to output the rendered response.

27. The system of claim 19, wherein to process the received digitized audio stream the server is further configured to:

convert the received digitized audio stream to a text stream; and

analyze the text stream to determine a meaning of the text stream.