Multimodal transmission of packetized data
First Claim
1. A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
- a natural language processor component executed by a data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client computing device;
the natural language processor component to parse the input audio signal to identify a request;
a direct action application programming interface of the data processing system to generate, based the request, a first action data structure;
a content selector component executed by the data processing system to receive at least the request to select a content item;
an interface management component of the data processing system to;
poll a plurality of interfaces to identify a first candidate interface and a second candidate interface;
determine a first interface parameter for the first candidate interface and a second interface parameter for the second candidate interface;
determine a first resource utilization value for the first candidate interface based on the first interface parameter and a second resource utilization value for the second candidate interface based the second interface parameter;
select the first candidate interface as a selected interface to present the content item based at least on the first interface parameter, the second interface parameter, the first resource utilization, and the second resource utilization;
provide the first action data structure to the client computing device for rendering as audio output from the client computing device; and
transmit the content item for rendering at the selected interface.
2 Assignments
0 Petitions
Accused Products
Abstract
A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify first and second candidate interfaces, and respective resource utilization values. The interface management component can select, based on the resource utilization values, the first candidate interface to present the content item. The interface management component can provide the first action data structure to the client computing device for rendering as audio output, and can transmit the content item converted for a first modality to deliver the content item for rendering from the selected interface.
-
Citations
20 Claims
-
1. A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
-
a natural language processor component executed by a data processing system to receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client computing device; the natural language processor component to parse the input audio signal to identify a request; a direct action application programming interface of the data processing system to generate, based the request, a first action data structure; a content selector component executed by the data processing system to receive at least the request to select a content item; an interface management component of the data processing system to; poll a plurality of interfaces to identify a first candidate interface and a second candidate interface; determine a first interface parameter for the first candidate interface and a second interface parameter for the second candidate interface; determine a first resource utilization value for the first candidate interface based on the first interface parameter and a second resource utilization value for the second candidate interface based the second interface parameter; select the first candidate interface as a selected interface to present the content item based at least on the first interface parameter, the second interface parameter, the first resource utilization, and the second resource utilization; provide the first action data structure to the client computing device for rendering as audio output from the client computing device; and transmit the content item for rendering at the selected interface. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A method of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
-
receiving, by a natural language processor component of a data processing system, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a client computing device; parsing, by the natural language processor component, the input audio signal to identify a request; generating, by a direct action application programming interface of the data processing system and based on the request, a first action data structure; selecting, by a content selector component of the data processing system, a content item based on the request; polling, by an interface management component of the data processing system, a plurality of interfaces to identify a first candidate interface and a second candidate interface; determining, by the interface management component of the data processing system, a first interface parameter for the first candidate interface and a second interface parameter for the second candidate interface; determining, by the interface management component of the data processing system, a first resource utilization value for the first candidate interface based on the first interface parameter and a second resource utilization value for the second candidate interface based the second interface parameter; selecting, by the interface management component of the data processing system, the first candidate interface as a selected interface to present the content item based at least on the first interface parameter, the second interface parameter, the first resource utilization, and the second resource utilization; providing, by the interface management component of the data processing system, the first action data structure to the client computing device for rendering as audio output from the client computing device; and transmitting, by the interface management component of the data processing system, the content item for rendering at the selected interface. - View Dependent Claims (19, 20)
-
Specification