System, device, and method for securing voice authentication and end-to-end speech interaction

• US 9,124,386 B2
• Filed: 09/28/2012
• Issued: 09/01/2015
• Est. Priority Date: 09/28/2012
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Dadu, Saurabh, Rishi, Karthik K., Prakash, Gyan, Poornachandran, Rajesh
• Primary Examiner(s): Zecher, Dede
• Assistant Examiner(s): MCCOY, RICHARD ANTHONY
• Application Number:
  US13/631,278
• Publication Number:
  US 20140093083A1
• Time in Patent Office: 1,068 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: H04K 1/00 Secret communication

Abstract

A method, device, and system for secure end-to-end audio recognition is disclosed. A client device launches an application that connects with a server. The client device and server exchange cryptographic keys and establish a secure connection and a shared cryptographic key. The server transmits an encrypted audio prompt to the client device. The client device decrypts the encrypted audio prompt and stores the decrypted audio prompt in secure memory inaccessible to the operating system using an audio engine of the client device. The audio engine then retrieves the audio and renders it for the user through the speakers of the client device. The client device captures the user'"'"'s audio response with a microphone and stores the audio response in the secure memory. The stored audio response is encrypted and transmitted to the server.

Claims

1. The A computing device for establishing secure voice authentication, the computing device comprising:

a central processing unit;

a main memory that is accessible by the central processing unit;

a secure memory that is inaccessible by the central processing unit;

a speaker;

a microphone;

a communication module to receive an encrypted audio prompt from a server;

a security engine to decrypt the encrypted audio prompt and store the decrypted audio prompt in the secure memory;

an audio engine to (i) retrieve the decrypted audio prompt from the secure memory, (ii) render the decrypted audio prompt on the speaker, (iii) capture an audio response generated by the microphone, and (iv) store the captured audio response in the secure memory; and

a command recognition module to correlate the captured audio response with one or more registered audio commands of the computing device,

wherein the security engine to further (i) retrieve the captured audio response from the secure memory, (ii) encrypt the audio response, and (iii) store the encrypted audio response in the main memory,

wherein the communication module to further transmit the encrypted audio response to the server.

2. The computing device of claim 1, wherein the audio engine comprises an audio co-processor, different from the central processing unit, to:

retrieve the decrypted audio prompt from the secure memory;

render the decrypted audio prompt on the speaker;

capture the audio response generated by the microphone; and

store the captured audio response in the secure memory.

3. The computing device of claim 1, wherein the security engine comprises a security co-processor, different from the central processing unit, to:

decrypt the encrypted audio prompt;

store the decrypted audio prompt in the secure memory;

retrieve the captured audio response from the secure memory;

encrypt the audio response; and

store the encrypted audio response in the main memory.

4. The computing device of claim 1, wherein the audio engine further comprises an audio render pipeline to render the decrypted audio prompt on the speaker in response to the audio render pipeline decoding the decrypted audio prompt.

5. The computing device of claim 1, wherein the audio engine further comprises an audio capture pipeline to store the captured audio response in the secure memory in response to the audio capture pipeline encoding the captured audio response.

14. The computing device of claim 2, wherein the audio co-processor is further to:

establish an audio capture pipeline inaccessible by the central processing unit to securely transfer a first audio data captured by the microphone from the microphone to the secure memory and

establish an audio render pipeline inaccessible by the central processing unit to securely transfer a second audio data to be rendered on the speaker from the secure memory to the speaker,

wherein the first audio data in the audio capture pipeline and the second audio data in the audio render pipeline are inaccessible by the central processing unit.

6. One or more non-transitory machine-readable storage media comprising a plurality of instructions stored thereon that, in response to being executed, result in a computing device:

receiving an encrypted audio prompt from a server, the audio prompt requesting a response from a user of the client computing device;

decrypting, using a security engine, the encrypted audio prompt;

storing, using the security engine, the decrypted audio prompt in a secure memory that is inaccessible by a central processing unit;

retrieving, with an audio engine, the decrypted audio prompt from the secure memory;

rendering, using the audio engine, the decrypted audio prompt on a speaker;

capturing, with the audio engine, an audio response generated by a microphone;

storing, using the audio engine, the captured audio response in the secure memory;

correlating the audio response with a registered audio command;

encrypting the audio response using the security engine;

storing, using the security engine, the encrypted audio response in a memory that is accessible by the central processing unit; and

transmitting the encrypted audio response to the server.

7. The one or more non-transitory machine-readable storage media of claim 6, wherein receiving the encrypted audio prompt from the server comprises receiving the encrypted audio prompt with an audio recognition application of the computing device.

8. The one or more non-transitory machine readable storage media of claim 6, wherein decrypting the encrypted audio prompt comprises decrypting the encrypted audio prompt using a cryptographic key shared with the server.

9. The one or more non-transitory machine-readable storage media of claim 6, wherein encrypting the audio response comprises encrypting the audio response using a cryptographic key shared with the server.

10. The one or more non-transitory machine readable storage media of claim 6, wherein the plurality of instructions further result in the computing device:

encoding, with the audio co-processor, the captured audio response, wherein storing the captured audio response in the secure memory comprises storing the encoded captured audio response; and

decoding, with the audio co-processor, the decrypted audio prompt, wherein rendering the decrypted audio prompt on the speaker comprises rendering the decoded decrypted audio prompt.

11. The one or more non-transitory machine-readable storage media of claim 6, wherein:

retrieving the decrypted audio prompt comprises retrieving, with an audio co-processor different from the central processing unit, the decrypted audio prompt from the secure memory;

rendering the decrypted audio prompt comprises rendering, by the audio-coprocessor, the decrypted audio prompt on a speaker;

capturing the audio response comprises capturing, by the audio co-processor, an audio response generated by a microphone; and

storing the captured audio response comprises storing, by the audio co-processor, the captured audio response in the secure memory.

12. The one or more machine-readable storage media of claim 6, wherein the plurality of instructions further result in the computing device decoding, with the audio engine, the decrypted audio prompt, wherein rendering the decrypted audio prompt on the speaker comprises rendering the decoded decrypted audio prompt.

13. The one or more non-transitory machine-readable storage media of claim 8, wherein the plurality of instructions further result in the computing device:

generating a private-public key pair comprising a client public key and a client private key;

transmitting the client public key, with a public key certificate signed by a private key corresponding with a public key stored in a public key database accessible to the server, to the server;

receiving, from the server, the shared cryptographic key encrypted with the client public key; and

decrypting the shared cryptographic key using the client private key.

15. The one or more non-transitory machine-readable storage media of claim 11, wherein the plurality of instructions further result in the computing device:

establishing a secure audio render pipeline inaccessible by the central processing unit by which to transfer the decrypted audio prompt from the secure memory to the speaker; and

establishing a secure audio capture pipeline inaccessible by the central processing unit by which to transfer the audio response from the microphone to the secure memory, wherein audio data in the secure audio capture pipeline and the secure audio render pipeline are inaccessible by the central processing unit.

16. One or more non-transitory machine-readable storage media comprising a plurality of instructions stored thereon that, in response to being executed, result in a computing device:

capturing, with an audio co-processor different from a central processing unit of the computing device, audio data generated by a microphone;

storing, using the audio co-processor, the captured audio data in a secure memory of the audio co-processor, wherein the secure memory is inaccessible by the central processing unit;

retrieving, with a security co-processor different from the central processing unit, the captured audio data from the secure memory of the audio co-processor;

encrypting, using the security engine co-processor, the retrieved audio data; and

storing the encrypted audio data in a system memory that is accessible by the central processing unit.

17. The one or more non-transitory machine-readable storage media of claim 16, wherein the plurality of instructions further result in the computing device establishing a secure audio capture pipeline inaccessible by the central processing unit by which to transfer the audio data generated by the microphone from the microphone to the secure memory, wherein audio data in the secure audio capture pipeline is inaccessible by the central processing unit.

18. One or more non-transitory machine-readable storage media comprising a plurality of instructions stored thereon that, in response to being executed, result in a computing device:

receiving, with an audio co-processor different from a central processing unit of the computing device, encrypted audio data from an application executed on the computing device;

decrypting, using a security co-processor different from the central processing unit, the encrypted audio data;

storing, using the security co-processor, the decrypted audio data in a secure memory of the audio co-processor, wherein the secure memory is inaccessible by a central processing unit;

retrieving, with the audio co-processor, the decrypted audio data from the secure memory of the audio co-processor; and

rendering, using the audio co-processor, the decrypted audio data on a speaker.

19. The one or more non-transitory machine-readable storage media of claim 18, wherein the plurality of instructions further result in the computing device establishing a secure audio render pipeline inaccessible by the central processing unit by which to transfer the decrypted audio data from the secure memory to the speaker, wherein audio data in the secure audio render pipeline is inaccessible by the central processing unit.