Data structure pooling of voice activated data packets

• US 10,423,621 B2
• Filed: 06/26/2018
• Issued: 09/24/2019
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bhaya, Gaurav, Stets, Robert
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US16/018,750
• Publication Number:
  US 20180308493A1
• Time in Patent Office: 455 Days
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

Systems and methods of voice activated thread management in a voice activated data packet based environment are provided. A natural language processor (“NLP”) component can receive and parse data packets comprising a first input audio signal to identify a first request and a first trigger keyword. A direct action application programming interface (“API”) can generate a first action data structure with a parameter defining a first action. The NLP component can receive and parse a second input audio signal to identify a second request and a second trigger keyword, and can generate a second action data structure with a parameter defining a second action. A pooling component can generate the first and second action data structures into a pooled data structure, and can transmit the pooled data structure to a service provider computing device to cause it device to perform an operation defined by the pooled data structure.

Claims

1. A system to consolidate voice activated threads in a voice activated data packet based computer network environment, comprising:

a natural language processor component executed by a data processing system to receive, via an interface of the data processing sytem, data packets comprising a first input audio signal detected by a sensor of a first client computing device;

the natural language processor component to parse the first input audio signal to identify a first request and a first trigger keyword corresponding to the first request;

a direct action application programming interface (“

API”

) of the data processing system to generate, based on the first trigger keyword and in response to the first request, a first action data structure with a parameter defining a first action;

the natural language processor component to receive, via the interface of the data processing system, data packets comprising a second input audio signal detected by a sensor of a second client computing device, and to parse the second input audio signal to identify a second request and a second trigger keyword corresponding to the second request;

the direct action API to generate, based on the second trigger keyword and in response to the second request, a second action data structure with a parameter defining a second action; and

a pooling component of the data processing system to;

determine based on a comparison of the first action with the second action a level of overlap between the first action data structure and the second action data structure;

generate, based on the comparison the level of overlap, the first data structure;

generate, based on the comparison the level of overlap, the first action data structure, and the second action data structure, a pooled data structure that consolidates the first action data structure and the second action data structure; and

transmit, via a computer network, the pooled data structure to a service provider computing device to cause the service provider computing device to perform an operation defined by the pooled data structure and corresponding to the first action and the second action.

2. The system of claim 1, comprising the data processing system to:

apply a heuristic technique to the first action data structure and the second action data structure to determine a pooling parameter that indicates the level of overlap between the first action data structure and the second action data structure.

3. The system of claim 1, comprising the data processing system to:

merge the first action data structure with the second action data structure to generate the pooled data structure.

4. the system of claim 1, comprising the data processing system to:

bypass execution of the first action data structure and the second action data structure responsive to generation of the pooled data structure.

5. The system of claim 1, comprising the data processing system to:

perform a plurality of actions via the pooled data structure.

6. The system of claim 1, comprising the data processing system to:

establish a session with the service provider computing device to execute to pooled data structure.

7. The system of claim 1, comprising the data processing system to:

establish a session with the service provider computing device that bypasses the first client computing device and the second client computing device to execute the pooled data structure.

8. The system of claim 1, wherein the data processing system comprises or interfaces with a voice activated assistant.

9. The system of claim 1, comprising the data processing system to:

determine the first action data structure indicates a first account and a first location;

determine the second action data structure indicates a second account different from the first account and the first location; and

generate the pooled data structure responsive to the first location indicated by both the first action data structure and the second action data structure.

10. The system of claim 1, wherein the level of overlap indicates a similarity metric between the first action data structure and the second action data structure.

11. The system of claim 1, wherein the first action includes a first plurality of sub-actions.

12. The system of claim 1, wherein the parameter defining the first action and the parameter defining the second action each indicate location data.

13. The system of claim 1, wherein the parameter defining the first action and the parameter defining the second action each identify common subject matter.

14. A method of consolidating voice activated threads in a voice activated data packet based computer network environment, comprising:

receiving, by a data processing system comprising one or more processors and memory, data packets comprising a first input audio signal detected by a sensor of a first client computing device;

parsing, by the data processing system, the first input audio signal to identify a first request and a first trigger keyword corresponding to the first request;

generating, by the data processing system, based on the first trigger keyword and in response to the first request, a first action data structure with the parameter defining a first action;

receiving, by the data processing system, data packets comprising a second input audio signal detected by a sensor of a second client computing device;

parsing, by the data processing system, the second input audio signal to identify a second request and a second trigger keyword corresponding to the second request;

generating, by the data processing system based on the second trigger keyword and in response to the second request, a second action data structure with a parameter defining a second action;

determining, by the data processing system, based on comparison of the first action with the second action a level of overlap between the first action data structure and the second action data structure;

generating, by the data processing system, based on the comparison and the level of overlap, the first action data structure, and the second action data structure a pooled data structure that consolidates the first action data structure and the second action data structure; and

transmitting, by the data processing system via a computer network, the pooled data structure to a service provider computing device to cause the service provider computing device to perform an operation defined by the pooled data structure and corresponding to the first action and the second action.

15. The method of claim 14, comprising:

determining, via application of a heuristic technique to the first action data structure and the second action data structure, a pooling parameter that indicates the level of overlap between the first action data structure and the second action data structure.

16. The method of claim 14, comprising:

merging the first action data structure with the second action data structure to generate the pooled data structure.

17. The method of claim 14, comprising:

bypassing execution of the first action data structure and the second action data structure responsive to generation of the pooled data structure.

18. The method of claim 14, comprising:

performing a plurality of actions via the pooled data structure.

19. The method of claim 14, comprising:

establishing a session with the service provider computing device to execute the pooled data structure.

20. The method of claim 14, comprising:

establishing a session with the service provider computing device that bypasses the first client computing device and the second client computing device to execute the pooled data structure.