Always-on audio control for mobile device

• US 10,431,224 B1
• Filed: 04/29/2019
• Issued: 10/01/2019
• Est. Priority Date: 11/12/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Millet, Timothy J., Gulati, Manu, Culbert, Michael F.
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US16/397,057
• Publication Number:
  US 20190287532A1
• Time in Patent Office: 155 Days
• CPC Class Codes: G06F 1/32 Means for saving power
• CPC Class Codes: G06F 1/3228 Monitoring task completion, e.g. by use of idle timers, stop commands or wait commands
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

In an embodiment, an integrated circuit may include one or more CPUs, a memory controller, and a circuit configured to remain powered on when the rest of the SOC is powered down. The circuit may be configured to receive audio samples from a microphone, and match those audio samples against a predetermined pattern to detect a possible command from a user of the device that includes the SOC. In response to detecting the predetermined pattern, the circuit may cause the memory controller to power up so that audio samples may be stored in the memory to which the memory controller is coupled. The circuit may also cause the CPUs to be powered on and initialized, and the operating system (OS) may boot. During the time that the CPUs are initializing and the OS is booting, the circuit and the memory may be capturing the audio samples.

Claims

1. An integrated circuit comprising:

one or more processors;

at least one memory controller; and

a first circuit coupled to the one or more processors and to the memory controller, wherein;

the first circuit is configured to receive audio samples captured by one or more audio input devices and to detect a predetermined pattern in the audio samples during a time that the one or more processors and the memory controller are powered down;

the first circuit is configured to cause the memory controller and the one or more processors to power up responsive to detecting the predetermined pattern, wherein a first delay to initialize the memory controller is less than a second delay to prepare the one or more processors to process the audio samples; and

the first circuit is configured to buffer the audio samples that match the pattern and subsequently-received samples during the first delay, write the audio samples that match the predetermined pattern and subsequently-received samples to memory through the memory controller subsequent to the first delay, and write additional samples received between an end of the first delay and an end of the second delay to the memory through the memory controller.

2. The integrated circuit as recited in claim 1 wherein the buffer is configured to store a number of the audio samples sufficient to cover the first delay without loss of audio samples.

3. The integrated circuit as recited in claim 1 wherein the buffer is configured to store a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples during the first delay.

4. The integrated circuit as recited in claim 1 wherein the first delay includes time to power up the memory controller and to restore one or more memory controller parameters by the first circuit to program the memory controller for operation with the memory.

5. The integrated circuit as recited in claim 1 wherein the second delay includes time to power up the one or more processors, reset the one or more processors, and boot an operating system on the one or more processors.

6. The integrated circuit as recited in claim 1 wherein the one or more processors are configured to access the audio samples in the memory to verify the match subsequent to the second delay.

7. The integrated circuit as recited in claim 1 wherein the predetermined pattern represents sound comprising a verbal utterance of one or more words.

8. The integrated circuit as recited in claim 7 wherein the predetermined pattern is captured from a user'"'"'s voice.

9. The integrated circuit as recited in claim 1 wherein the predetermined pattern represents a non-verbal sound.

10. The integrated circuit as recited in claim 1 wherein the first circuit comprises a processor configured to execute a plurality of instructions to perform at least the detecting of the predetermined pattern in the audio samples and causing the power up of the processors and the memory controller.

11. The integrated circuit as recited in claim 1 wherein the first circuit comprises hardware circuitry configured to detect the predetermined pattern in the audio samples.

12. A system comprising:

an audio input device;

an audio coder/decoder (codec) coupled to the audio input device and configured to generate audio samples from sound detected by the audio input device;

a memory; and

an integrated circuit coupled to the audio codec and the memory, wherein the integrated circuit includes an audio filter, one or more processors, and a memory controller coupled to the memory;

the audio filter is configured to detect a predetermined pattern in the audio samples from the audio codec during a time that the memory controller and the one more processors are powered down and is configured to cause the memory controller and the one or more processors to power up responsive to detecting the predetermined pattern, wherein a first delay to initialize the memory controller is less than a second delay to prepare the one or more processors to process the audio samples; and

the audio filter is configured to buffer the audio samples that match the pattern and subsequently-received samples during the first delay, write the audio samples that match the predetermined pattern and subsequently-received samples to memory through the memory controller subsequent to the first delay, and write additional samples received between an end of the first delay and an end of the second delay to the memory through the memory controller.

13. The system as recited in claim 12 wherein the buffer is configured to store a number of the audio samples sufficient to cover the first delay without loss of audio samples.

14. The system as recited in claim 12 wherein the buffer is configured to store a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples during the first delay.

15. The system as recited in claim 12 wherein the first delay includes time to power up the memory controller and to restore one or more memory controller parameters by the first circuit to program the memory controller for operation with the memory.

16. The system as recited in claim 12 wherein the second delay includes time to power up the one or more processors, reset the one or more processors, and boot an operating system on the one or more processors.

17. A method comprising:

during a time that the one or more processors and a memory controller in an integrated circuit are powered down, monitoring audio samples in a first circuit within the integrated circuit that remains powered;

detecting a predetermined pattern in the audio samples in the first circuit;

requesting power up of the memory controller and the one or more processors in response to the detecting, wherein a first delay to initialize the memory controller is less than a second delay to prepare the one or more processors to process the audio samples;

buffering the audio samples and subsequently-received audio samples in a buffer in the first circuit during the first delay;

writing the audio samples that match the predetermined pattern and subsequently-received samples to memory through the memory controller subsequent to the first delay; and

writing additional samples received between an end of the first delay and an end of the second delay to the memory through the memory controller.

18. The method as recited in claim 17 wherein the buffering comprises storing a plurality of the audio samples, wherein a number of the plurality of audio samples is sufficient to store the audio samples that match the predetermined pattern and the subsequently-received samples during the first delay.

19. The method as recited in claim 17 wherein the first delay includes time to power up the memory controller and to restore one or more memory controller parameters by the first circuit to program the memory controller for operation with the memory.

20. The method as recited in claim 17 wherein the second delay includes time to power up the one or more processors, reset the one or more processors, and boot an operating system on the one or more processors.