Authentication of packetized audio signals

• US 10,541,998 B2
• Filed: 01/05/2018
• Issued: 01/21/2020
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bhaya, Gaurav, Stets, Robert
• Primary Examiner(s): Lagor, Alexander
• Assistant Examiner(s): Jones, William B
• Application Number:
  US15/863,042
• Publication Number:
  US 20180191713A1
• Time in Patent Office: 746 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/34 involving the use of external additional devices, e.g. dongles or smart cards
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2115 Third party
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04W 12/63 Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 4/025 using location based information parameters

Abstract

The present disclosure is generally directed a data processing system for authenticating packetized audio signals in a voice activated computer network environment. The data processing system can improve the efficiency and effectiveness of auditory data packet transmission over one or more computer networks by, for example, disabling malicious transmissions prior to their transmission across the network. The present solution can also improve computational efficiency by disabling remote computer processes possibly affected by or caused by the malicious audio signal transmissions. By disabling the transmission of malicious audio signals, the system can reduce bandwidth utilization by not transmitting the data packets carrying the malicious audio signal across the networks.

Claims

1. A system to authenticate packetized audio signals in a voice activated computer network environment, comprising:

a data processing system comprising at least one processor and memory;

a natural language processor component executed by the data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

the natural language processor component to parse the input audio signal to identify a request and a trigger keyword corresponding to the request;

a direct action application programming interface of the data processing system to generate, based on the trigger keyword, a first action data structure responsive to the request;

a network security appliance to authenticate the input audio signal detected by the sensor of the client device based on a characteristic of the input audio signal and the first action data structure generated by the direct action application programming interface;

the direct action application programming interface to identify an account based on the input audio signal authenticated by the network security appliance, and transmit, to a third-party provider device, the first action data structure responsive to the network security appliance authenticating the input audio signal based on the characteristic of the input audio signal and the first action data structure, receipt of the first action data structure by the third-party provider device causing the third-party provider device to execute the first action data structure;

receive second data packets comprising a second input audio signal detected by the sensor of the client device;

generate an alarm condition responsive to a second characteristic of the second input audio signal not matching a parameter associated with the first action data structure; and

transmit, responsive to generation of the alarm condition, an instruction to the third-party provider device to terminate a communication session associated with, or execution of, the first action data structure.

2. The system of claim 1, comprising:

the network security appliance to authenticate the input audio signal based on the characteristic of the input audio signal, the characteristic including at least one a voiceprint.

3. The system of claim 1, comprising the network security appliance to:

measure a spectral component of the input audio signal to identify a voiceprint;

compare the spectral component with a stored voiceprint generated during a setup phase;

and authenticate the input audio signal based on the comparison of the spectral component with the stored voiceprint.

4. The system of claim 1, comprising the network security appliance to:

determine the characteristic of the input audio signal based on a location of the client device; and

authenticate the input audio signal based on the location of the client device.

5. The system of claim 1, comprising the network security appliance to:

determine the characteristic of the input audio signal with a physical authentication device; and

authenticate the input audio signal based on a response from the physical authentication device.

6. The system of claim 1, comprising the network security appliance to:

determine the characteristic of the input audio signal based on a number of voices detected in the input audio signal; and

authenticate the input audio signal based on the number of voices detected.

7. The system of claim 1, comprising the network security appliance to:

receive a third input audio signal detected by the sensor of the client device;

detect, based on a characteristic of the third input audio signal, a second alarm condition; and

prevent, responsive to the second alarm condition, execution of a third action data structure based on the third input audio signal.

8. The system of claim 1, comprising the data processing system to:

perform a search based on the first action data structure and the characteristic of the input audio signal authenticated by the network security appliance.

9. The system of claim 1, comprising the data processing system to:

identify the account based on the characteristic of the input audio signal;

identify a preference associated with the account; and

perform a search based on the preference.

10. The system of claim 1, comprising the data processing system to:

identify the account based on the characteristic of the input audio signal;

identify a first preference associated with the account;

perform a first search based on the first preference;

identify a second account based on the characteristic of the second input audio signal detected by the sensor of the client device;

identify a second preference associated with the second account; and

perform a second search based on the second preference.

11. A method of authenticating packetized audio signals in a voice activated computer network environment, comprising:

receiving, by a data processing system comprising at least one processor and memory that executes a natural language processor component, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

parsing, by the natural language processor component, the input audio signal to identify a request and a trigger keyword corresponding to the request;

generating, by a direct action application programming interface of the data processing system, based on the trigger keyword, a first action data structure responsive to the request;

authenticating, by a network security appliance, the input audio signal detected by the sensor of the client device based on a characteristic of the input audio signal and the first action data structure generated by the direct action application programming interface; and

identifying, by the data processing system, an account based on the input audio signal authenticated by the network security appliance;

transmitting, by the data processing system, to a third-party provider device, the first action data structure responsive to the network security appliance authenticating the input audio signal based on the characteristic of the input audio signal and the first action data structure, receipt of the first action data structure by the third-party provider device causing the third-party provider device to execute the first action data structure;

receiving, by the data processing system, second data packets comprising a second input audio signal detected by the sensor of the client device;

generating, by the data processing system, an alarm condition responsive to a second characteristic of the second input audio signal not matching a parameter associated with the first action data structure; and

transmitting, by the data processing system, responsive to generation of the alarm condition, an instruction to the third-party provider device to terminate a communication session associated with, or execution of, the first action data structure.

12. The method of claim 11, comprising:

authenticating, by the network security appliance, the input audio signal based on the characteristic of the input audio signal, the characteristic including at least one a voiceprint.

13. The method of claim 11, comprising:

measuring a spectral component of the input audio signal to identify a voiceprint;

comparing the spectral component with a stored voiceprint generated during a setup phase; and

authenticating the input audio signal based on the comparing of the spectral component with the stored voiceprint.

14. The method of claim 11, comprising:

determining the characteristic of the input audio signal based on a location of the client device; and

authenticating the input audio signal based on the location of the client device.

15. The method of claim 11, comprising:

determining the characteristic of the input audio signal with a physical authentication device; and

authenticating the input audio signal based on a response from the physical authentication device.

16. The method of claim 11, comprising:

determining the characteristic of the input audio signal based on a number of voices detected in the input audio signal; and

authenticating the input audio signal based on the number of voices detected.

17. The method of claim 11, comprising:

receive a third input audio signal detected by the sensor of the client device;

detect, based on a characteristic of the third input audio signal, a second alarm condition; and

prevent, responsive to the second alarm condition, execution of a third action data structure based on the third input audio signal.

18. The method of claim 11, comprising:

performing a search based on the first action data structure and the characteristic of the input audio signal authenticated by the network security appliance.

19. The method of claim 11, comprising:

identifying the account based on the characteristic of the input audio signal;

identifying a preference associated with the account; and

performing a search based on the preference.

20. The method of claim 11, comprising:

identifying the account based on the characteristic of the input audio signal;

identifying a first preference associated with the account;

performing a first search based on the first preference;

identifying a second account based on the characteristic of the second input audio signal detected by the sensor of the client device;

identifying a second preference associated with the second account; and

performing a second search based on the second preference.