Mechanism for retrieval of previously captured audio

• US 10,452,339 B2
• Filed: 10/16/2015
• Issued: 10/22/2019
• Est. Priority Date: 06/05/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moore, Jeffrey C., Powell, Richard M., Powers, Alexander C., Guetta, Anthony J.
• Primary Examiner(s): Elahee, Md S
• Assistant Examiner(s): Siegel, David
• Application Number:
  US14/885,908
• Publication Number:
  US 20160357508A1
• Time in Patent Office: 1,467 Days
• CPC Class Codes: G06F 13/20 for access to input/output bus
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G11B 2020/10666 Ring buffers, e.g. buffers wherein an iteratively progressing read or write pointer moves back to the beginning of the buffer when reaching the last storage cell

Abstract

In one embodiment, a device-side audio handling input/output unit (DIO) of a microphone writes audio data generated by the microphone into a ring buffer. A system-side audio handling I/O unit (SIO) receives a request from a software program to consume past audio data from the ring buffer. The SIO provides the past audio data from the ring buffer to the software program. Other embodiments are also described and claimed.

Claims

1. A method performed by a computer system, comprising:

writing, by a device-side audio handling input/output unit (DIO) of a hardware device, audio data generated by the hardware device at a DIO position into a ring buffer;

receiving, by a system-side audio handling input/output unit (SIO), a request from a software program to consume past audio data from the ring buffer;

providing, by the SIO, the past audio data from the ring buffer to the software program, starting from an SIO position that is behind the DIO position in the ring buffer;

putting, by the SIO, a client thread of the software program to sleep until the DIO position is ahead of the SIO position by at least a client-side offset; and

when the DIO position is ahead of the SIO position by at least the client-side offset

awaking, by the SIO, the client thread of the software program, and

providing, by the SIO, the audio data from the ring buffer to the software program, by starting to provide the audio data from the SIO position within the ring buffer after providing the past audio data and catching up to real-time audio data being written into the ring buffer.

2. The method of claim 1, further comprising:

providing, by the SIO, an amount of the past audio data available in the ring buffer to the software program in response to receiving a request from the software program to determine how much of the past audio data is available in the ring buffer.

3. The method of claim 1, further comprising:

providing, by the SIO, an indication to the software program of whether the past audio data can be accessed in response to receiving a request from the software program to determine whether the past audio data can be accessed.

4. The method of claim 1, wherein the request to consume the past audio data from the ring buffer specifies a time value, wherein the time value corresponds to a position in the ring buffer from which to start providing the past audio data.

5. The method of claim 1, wherein the hardware device is a microphone and the audio data is sound captured by the microphone.

6. The method of claim 1, wherein the software program is a voice command application or a music recognition application.

7. The method of claim 1, wherein the software program and the SIO is executed by a main processor of the computer system and the DIO of the hardware device is executed by an auxiliary processor of the computer system, wherein the auxiliary processor is active while the main processor is deactivated.

8. The method of claim 1, wherein the DIO is executed in a kernel space of an operating system and wherein the ring buffer is in the kernel space of the operating system.

9. A computer system comprising:

one or more processors; and

a non-transitory computer readable storage medium having instructions stored therein, the instructions when executed by the one or more processors cause the computer system to

write, by a device-side audio handling input/output unit (DIO) of a hardware device, audio data generated by the hardware device at a DIO position within a ring buffer, wherein the DIO enables higher layer programs to access hardware functions of the hardware device without needing to know details of the hardware functions,

receive, by a system-side audio handling input/output unit (SIO), a request from a software program executed by the computer system to consume past audio data from the ring buffer,

provide, by the SIO, the past audio data starting from a SIO position, which is behind the DIO position, within the ring buffer to the software program,

put, by the SIO, a client thread of the software program to sleep until the DIO position is ahead of the SIO position by at least a client-side offset; and

when the DIO position is ahead of the SIO position by at least the client-side offset

awake, by the SIO, the client thread of the software program; and

provide, by the SIO, the audio data from the ring buffer to the software program by

starting to provide the audio data from the SIO position within the ring buffer to the software program after providing the past audio data and catching up to the real-time audio data being written into the ring buffer.

10. The computer system of claim 9, wherein the one or more processors include a main processor and an auxiliary processor, wherein the main processor is operable to execute the software program and the SIO, and the auxiliary processor is operable to execute the DIO of the hardware device, and wherein the auxiliary processor is operable to execute the DIO of the hardware device when the main processor is deactivated.

11. A computer system having audio-based user input capability, comprising:

a ring buffer;

a microphone; and

one or more processors to execute a device-side audio handling input/output unit (DIO) and a system-side audio handling input/output unit (SIO),

wherein the DIO is a device driver of the microphone that writes digital audio data produced by the microphone at a DIO position within the ring buffer,

wherein the SIO receives a request from a software program to consume past audio data from the ring buffer, wherein the SIO provides the past audio data starting from a SIO position, which is behind the DIO position, within the ring buffer to the software program, while the DIO continues to write the audio data being produced by the microphone at the DIO position within the ring buffer, and

wherein the SIO is to put a client thread of the software program to sleep until the DIO position is ahead of the SIO position by at least a client-side offset, and when the DIO position is ahead of the SIO position by at least the client-side offset, the SIO is to (i) awaken the client thread of the software program, and (ii) provide the audio data from the ring buffer to the software program, and

wherein the SIO provides the audio data from the SIO position within the ring buffer to the software program after providing the past audio data and catching up to real-time audio data being written into the ring buffer.

12. The computer system of claim 11, wherein the SIO provides an amount of the past audio data available in the ring buffer to the software program in response to receiving a request from the software program to determine how much of the past audio data is available in the ring buffer.

13. The computer system of claim 11, wherein the SIO provides an indication to the software program of whether the past audio data can be accessed in response to receiving a request from the software program to determine whether the past audio data can be accessed.

14. The computer system of claim 11, wherein the request to consume the past audio data from the ring buffer specifies a time value, wherein the time value corresponds to a position in the ring buffer from which to start providing the past audio data.

15. The computer system of claim 11, wherein the software program is a voice command application or a music recognition application.

16. The computer system of claim 11, wherein the software program and the SIO are executed by a main processor of the computer system and the DIO of the hardware device is executed by an auxiliary processor of the computer system, wherein the auxiliary processor is active while the main processor is deactivated.

17. The computer system of claim 11, wherein the DIO is executed in a kernel space of an operating system and wherein the ring buffer is in the kernel space of the operating system.

18. The computer system of claim 11 wherein the ring buffer, the microphone and a processor that executes the DIO are part of an in-vehicle infotainment unit, and another processor that executes the software program which is requesting to consume the past audio data and the SIO is in a handheld portable computer system communicatively linked to the in-vehicle infotainment unit.