Multimodal transmission of packetized data
First Claim
1. A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
- a data processing system comprising hardware circuitry and memory to;
receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a computing device;
parse the input audio signal to identify a request and a trigger keyword corresponding to the request;
generate, based on at least one of the request and the trigger keyword, a first action data structure;
select, based on at least one of the request and the trigger keyword, a content item via a real-time content selection process;
poll a plurality of interfaces to identify a first candidate interface and a second candidate interface within a threshold distance of the computing device;
determine a first resource utilization value for the first candidate interface and a second resource utilization value for the second candidate interface;
select, based on a comparison of the first resource utilization value and the second resource utilization value, the first candidate interface as a selected interface to present the content item;
provide the content item for delivery in a modality compatible with the selected interface selected based on the comparison of the first resource utilization value and the second resource utilization value; and
transmit the content item in the modality to deliver the content item for presentation via the selected interface.
2 Assignments
0 Petitions
Accused Products
Abstract
A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify first and second candidate interfaces, and respective resource utilization values. The interface management component can select, based on the resource utilization values, the first candidate interface to present the content item. The interface management component can provide the first action data structure to the client computing device for rendering as audio output, and can transmit the content item converted for a first modality to deliver the content item for rendering from the selected interface.
205 Citations
20 Claims
-
1. A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
-
a data processing system comprising hardware circuitry and memory to; receive, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a computing device; parse the input audio signal to identify a request and a trigger keyword corresponding to the request; generate, based on at least one of the request and the trigger keyword, a first action data structure; select, based on at least one of the request and the trigger keyword, a content item via a real-time content selection process; poll a plurality of interfaces to identify a first candidate interface and a second candidate interface within a threshold distance of the computing device; determine a first resource utilization value for the first candidate interface and a second resource utilization value for the second candidate interface; select, based on a comparison of the first resource utilization value and the second resource utilization value, the first candidate interface as a selected interface to present the content item; provide the content item for delivery in a modality compatible with the selected interface selected based on the comparison of the first resource utilization value and the second resource utilization value; and transmit the content item in the modality to deliver the content item for presentation via the selected interface. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A method of multi-modal transmission of packetized data in a voice activated data packet based computer network environment, comprising:
-
receiving, by a data processing system comprising hardware circuitry and memory, via an interface of the data processing system, data packets comprising an input audio signal detected by a sensor of a computing device; identifying, by the data processing system, from the input audio signal, a request and a trigger keyword corresponding to the request; generating, by the data processing system, based on at least one of and the trigger keyword, a first action data structure; selecting, based on at least one of the request and the trigger keyword, a content item via a real-time content selection process; polling, by the data processing system, a plurality of interfaces to identify a first candidate interface and a second candidate interface within a threshold distance of the computing device; determining, by the data processing system, a first resource utilization value for the first candidate interface and a second resource utilization value for the second candidate interface; selecting, based on a comparison of the first resource utilization value and the second resource utilization value, the first candidate interface as a selected interface to present the content item; converting the content item for delivery in a modality compatible with the first candidate interface selected based on the comparison of the first resource utilization value and the second resource utilization value; and transmitting the content item converted for the modality to deliver the content item for rendering from the selected interface. - View Dependent Claims (19, 20)
-
Specification