Authentication of packetized audio signals

• US 10,541,997 B2
• Filed: 12/30/2016
• Issued: 01/21/2020
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bhaya, Gaurav, Stets, Robert
• Primary Examiner(s): Lagor, Alexander
• Assistant Examiner(s): Jones, William B
• Application Number:
  US15/395,729
• Publication Number:
  US 20180191711A1
• Time in Patent Office: 1,117 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/34 involving the use of external additional devices, e.g. dongles or smart cards
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06F 2221/2115 Third party
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 17/02 Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
• CPC Class Codes: G10L 17/24 the user being prompted to utter a password or a predefined phrase
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04L 63/0861 using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04W 12/63 Location-dependent; Proximity-dependent
• CPC Class Codes: H04W 4/025 using location based information parameters

Abstract

The present disclosure is generally directed a data processing system for authenticating packetized audio signals in a voice activated computer network environment. The data processing system can improve the efficiency and effectiveness of auditory data packet transmission over one or more computer networks by, for example, disabling malicious transmissions prior to their transmission across the network. The present solution can also improve computational efficiency by disabling remote computer processes possibly affected by or caused by the malicious audio signal transmissions. By disabling the transmission of malicious audio signals, the system can reduce bandwidth utilization by not transmitting the data packets carrying the malicious audio signal across the networks.

Claims

1. A system to authenticate packetized audio signals in a voice activated computer network environment, comprising:

a data processing system comprising at least one processor and memory;

a natural language processor component executed by the data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

the natural language processor component to parse the input audio signal to identify a request and a trigger keyword corresponding to the request;

a direct action application programming interface of the data processing system to generate, based on the trigger keyword, a first action data structure responsive to the request, the first action data structure comprising one or more parameters packaged together to cause a third party provider device to perform an action;

a conversational application programming interface of the data processing system to establish, based on the first action data structure, a communication session between the third party provider device and the client device;

a network security appliance of the data processing system to;

compare a parameter of the first action data structure with a first characteristic of the input audio signal; and

generate a first alarm condition responsive to the first characteristic of the input audio signal not matching the parameter of the first action data structure;

a content selector component of the data processing system to select, responsive to generation of the first alarm condition and based on the trigger keyword, a content item for provision to the client device for authentication of the input audio signal;

the network security appliance to;

receive a second set of data packets from the client device, the second set of data packets comprising a response audio signal comprising a response to the content item provisioned to the client device for authentication of the input audio signal;

generate a second alarm condition responsive to a second characteristic of the response audio signal not matching a second parameter of the first action data structure; and

transmit, responsive to generation of the second alarm condition, an instruction to the third party provider device to disable the communication session established with the client device.

2. The system of claim 1, the network security appliance to:

determine the first characteristic of the input audio signal; and

determine the second characteristic of the response audio signal, wherein the first characteristic and the second characteristic comprise at least one of a voiceprint, a keyword, a number of voices detected, an identification of the client device, or a location of a source of the input audio signal.

3. The system of claim 1, wherein the first characteristic is different from the second characteristic.

4. The system of claim 1, the network security appliance to:

receive a location of a second client device;

determine a distance between a location of the client device and the location of the second client device; and

detect the first alarm condition based on the distance between a location of the client device and the location of the second client device.

5. The system of claim 4, the network security appliance to:

detect the first alarm condition based on the distance between a location of the client device and the location of the second client device being above a predetermined threshold.

6. The system of claim 1, wherein the content item comprises instructions for generating an auditory signal at the client device.

7. The system of claim 6, wherein the auditory signal comprises a security question.

8. The system of claim 1, the network security appliance to:

disable the first action data structure responsive to detection of the first alarm condition.

9. The system of claim 1, the content selector component to:

generate instructions to capture the second characteristic of the response audio signal in the content item.

10. The system of claim 1, the network security appliance to:

terminate the communication session established with the client device in response to an interaction with the content item.

11. The system of claim 1, the network security appliance to determine an amount of computational resources required to complete the request.

12. The system of claim 11, the network security appliance to set the first alarm condition responsive to the amount of computational resources being above a predetermined threshold.

13. The system of claim 1, the natural language processor component to parse the response audio signal to identify a passphrase.

14. The system of claim 13, the network security appliance to set the second alarm condition based on the passphrase not matching a stored passphrase.

15. The system of claim 13, wherein the passphrase is the second characteristic.

16. A method for authenticating packetized audio signals in a voice activated computer network environment, comprising:

receiving, by a natural language processor component executed by a data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

parsing, by the natural language processor component, the input audio signal to identify a request and a trigger keyword corresponding to the request;

generating, by a direct action application programming interface of the data processing system, based on the trigger keyword, a first action data structure responsive to the request, the first action data structure comprising one or more parameters packaged together to cause a third party provider device to perform an action;

generating, by a conversational application programming interface of the data processing system and based on the first action data structure, a communication session between the third party provider device and the client device;

comparing, by a network security appliance of the data processing system, a parameter of the first action data structure with a first characteristic of the input audio signal;

generating, by the network security appliance, a first to detect an alarm condition responsive to the first characteristic of the input audio signal not matching the parameter of the first action data structure;

selecting, by a content selector component executed by the data processing system and responsive to generation of the first alarm condition, a content item for provision to the client device for authentication of the input audio signal;

receiving, by the network security appliance, a second set of data packets from the client device, the second set of data packets comprising a response audio signal comprising a response to the content item provisioned to the client device for authentication of the input audio signal;

generating, by the network security appliance, a second alarm condition responsive to a second characteristic of the response audio signal not matching a second parameter of the first action data structure; and

transmitting, by the network security appliance, responsive to the generation of the second alarm condition, an instruction to the third party provider device to disable the communication session established with the client device.

17. The method of claim 16, comprising:

determining, by the network security appliance, the first characteristic of the input audio signal; and

determining, by the network security appliance, the second characteristic of the response audio signal, wherein the first characteristic and the second characteristic comprise at least one of a voiceprint, a keyword, a number of voices detected, an identification of the client device, or a location of a source of the input audio signal.

18. The method of claim 16, comprising:

receiving, by the network security appliance, a location of a second client device;

determining, by the network security appliance, a distance between a location of the client device and the location of the second client device; and

detecting, by the network security appliance, the alarm condition based on the distance between a location of the client device and the location of the second client device.

19. A system to authenticate packetized audio signals in a voice activated computer network environment, comprising:

a data processing system comprising at least one processor and memory;

a natural language processor component executed by the data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client device;

the natural language processor component to parse the input audio signal to identify a request and a trigger keyword corresponding to the request;

a direct action application programming interface of the data processing system to generate, based on the trigger keyword, a first action data structure responsive to the request;

a conversational application programming interface of the data processing system to establish, based on the first action data structure, a communication session between a third party provider device and the client device;

a network security appliance of the data processing system to;

compare a parameter of the first action data structure with a first characteristic of the input audio signal;

generate a first alarm condition responsive to the first characteristic of the input audio signal not matching the parameter of the first action data structure;

a content selector component executed by the data processing system to select, responsive to generation of the first alarm condition and based on the trigger keyword, a content item for provision to the client device for authentication of the input audio signal;

the network security appliance to;

receive a second set of data packets from the client device, the second set of data packets carrying a response audio signal comprising a response to the content item provisioned to the client device for authentication of the input audio signal;

generate a pass condition responsive to a second characteristic of the response audio signal matching with a second parameter of the first action data structure; and

transmit, based on the generation of the pass condition, an instruction to the third party provider device to continue the communication session established with the client device.

20. The system of claim 19, the network security appliance to:

determine the first characteristic of the input audio signal; and

determine the second characteristic of the response audio signal, wherein the first characteristic and the second characteristic comprise at least one of a voiceprint, a keyword, a number of voices detected, an identification of the client device, or a location of a source of the input audio signal and the second characteristic comprises a security question.