Always-on audio control for mobile device

• US 10,079,019 B2
• Filed: 12/17/2013
• Issued: 09/18/2018
• Est. Priority Date: 11/12/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Millet, Timothy J., Gulati, Manu, Culbert, Michael F.
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US14/109,101
• Publication Number:
  US 20150134331A1
• Time in Patent Office: 1,736 Days
• Field of Search: 704243
• CPC Class Codes: G06F 1/32 Means for saving power
• CPC Class Codes: G06F 1/3228 Monitoring task completion, e.g. by use of idle timers, stop commands or wait commands
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

In an embodiment, an integrated circuit may include one or more CPUs, a memory controller, and a circuit configured to remain powered on when the rest of the SOC is powered down. The circuit may be configured to receive audio samples from a microphone, and match those audio samples against a predetermined pattern to detect a possible command from a user of the device that includes the SOC. In response to detecting the predetermined pattern, the circuit may cause the memory controller to power up so that audio samples may be stored in the memory to which the memory controller is coupled. The circuit may also cause the CPUs to be powered on and initialized, and the operating system (OS) may boot. During the time that the CPUs are initializing and the OS is booting, the circuit and the memory may be capturing the audio samples.

Claims

1. An integrated circuit comprising:

one or more processors;

at least one memory controller; and

a first circuit coupled to the one or more processors and to the memory controller, wherein the first circuit is configured to remain powered up during times that the one or more processors and the memory controller are powered down, and wherein the first circuit is configured to receive audio samples captured by one or more audio input devices and to detect a predetermined pattern in the audio samples during a time that the one or more processors and the memory controller are powered down, and wherein the first circuit is configured to cause the memory controller and the one or more processors to power up responsive to detecting the predetermined pattern, and wherein the first circuit is configured to write the audio samples that match the predetermined pattern and subsequently-received samples to memory through the memory controller subsequent to powering up the memory controller in response to detecting the predetermined pattern, including writing audio samples to the memory through the memory controller during a time;

(1) after the memory controller and the one or more processors are powered up but the one or more processors are still booting an operating system;

(2) the memory controller is available to accept write operations; and

(3) prior to the one or more processors being ready to operate on the audio samples,

whereby the audio samples that match the predetermined pattern are available in the memory for the one or more processors to verify the match subsequent to powering up responsive to the first circuit detecting the match; and

wherein the first circuit comprises a buffer configured to store a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples until the memory controller is available to receive writes of the audio samples to the memory.

2. The integrated circuit as recited in claim 1 wherein the predetermined pattern represents sound comprising a verbal utterance of one or more words.

3. The integrated circuit as recited in claim 2 wherein the predetermined pattern is captured from a user'"'"'s voice.

4. The integrated circuit as recited in claim 1 wherein the predetermined pattern represents a non-verbal sound.

5. The integrated circuit as recited in claim 1 wherein the first circuit is configured to provide one or more memory controller parameters to the memory controller to program the memory controller for operation with the memory.

6. The integrated circuit as recited in claim 1 wherein the first circuit comprises a processor configured to execute a plurality of instructions to perform at least the detecting of the predetermined pattern in the audio samples and causing the power up of the processors and the memory controller.

7. The integrated circuit as recited in claim 1 wherein the first circuit comprises hardware circuitry configured to detect the predetermined pattern in the audio samples.

8. A system comprising:

an audio input device;

an audio coder/decoder (codec) coupled to the audio input device and configured to generate audio samples from sound detected by the audio input device;

a memory; and

an integrated circuit coupled to the audio codec and the memory, wherein the integrated circuit includes an audio filter, one or more processors, and a memory controller coupled to the memory, and wherein the audio filter is configured to detect a predetermined pattern in the audio samples from the audio codec during a time that the memory controller and the one more processors are powered down and is configured to cause the memory controller and the one or more processors to power up responsive to detecting the predetermined pattern, wherein the integrated circuit is configured to buffer the audio samples that match the pattern and subsequently-received audio samples until the memory controller is able to accept write operations to write the samples to the memory, and wherein the audio filter is configured to transmit write operations to the memory controller to write audio samples to memory, including during a time;

(1) after the memory controller and the one or more processors are powered up but the one or more processors are still booting an operating system;

(2) the memory controller is available to accept write operations; and

(3) prior to the one or more processors being ready to operate on the audio samples,

whereby the audio samples that match the pattern are available in the memory for the one or more processors to verify the match subsequent to powering up responsive to the audio filter detecting the match; and

wherein the audio filter comprises a buffer configured to store a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples until the memory controller is available to receive writes of the audio samples to the memory.

9. The system as recited in claim 8 wherein the audio filter is configured to supply a plurality of memory controller parameters to the memory controller to program the memory controller for operation, wherein the audio filter is configured to generate the write operations subsequent to programming the memory controller.

10. The system as recited in claim 8 wherein the memory is packaged with the integrated circuit.

11. The system as recited in claim 8 wherein the audio input device is one of a plurality of microphones included in the system, and wherein other ones of the plurality of microphones are disabled during times that the memory controller is powered down.

12. The system as recited in claim 8 wherein the one or more processors are configured to execute a plurality of instructions to verify that the predetermined pattern has been detected and to interpret the subsequently-received audio samples.

13. A method comprising:

powering down a central processing unit (CPU) complex and a memory controller in an integrated circuit;

during a time that the CPU complex and the memory controller are powered down, monitoring audio samples in a first circuit within the integrated circuit that remains powered;

detecting a predetermined pattern in the audio samples in the first circuit;

requesting power up of at least the memory controller in response to the detecting; and

ensuring that the audio samples and subsequently-received audio samples are not lost by buffering the audio samples and subsequently-received audio samples in the first circuit until the memory controller is available and writing the buffered audio samples to memory, including during a time;

(1) after the memory controller and the one or more processors are powered up but the one or more processors are still booting an operating system;

(2) the memory controller is available to accept write operations; and

(3) prior to the one or more processors being ready to operate,

whereby a continuous stream of audio samples are available for processing in the memory and the audio samples that match the predetermined pattern are available in the memory for the one or more processors to verify the match subsequent to powering up; and

wherein the first circuit comprises a buffer configured to store a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples until the memory controller is available to receive writes of the audio samples to the memory.

14. The method as recited in claim 13 further comprising requesting power up of the CPU complex.

15. The method as recited in claim 14 further comprising:

booting an operating system on the CPU complex; and

processing the buffered audio samples from the memory.

16. The method as recited in claim 15 wherein the processing comprises:

verifying that the predetermined pattern is detected; and

interpreting the subsequently-received audio samples.