System and method for processing multi-modal device interactions in a natural language voice services environment
First Claim
1. A method for processing one or more multi-modal device interactions, received from a user, in a natural language voice services environment that includes a plurality of components that handle requests relating to the multi-modal device interact, the method being implemented on a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:
- detecting, by the computer system, at least one multi-modal device interaction, wherein the multi-modal device interaction includes a non-voice interaction, from the user, with at least one of the plurality of components or an application associated with at least one of the plurality of components, and wherein the multi-modal device interaction further includes at least one natural language utterance, from the user, relating to the non-voice interaction;
determining, by the computer system, a context relating to the non-voice interaction and a context relating to the natural language utterance;
determining, by the computer system, an intent of the multi-modal device interaction based on the context relating to the non-voice interaction and the context of the natural language utterance;
generating, by the computer system, a request based on the determined intent;
obtaining, by the computer system, information indicating a capability of a component, from among the plurality of components, based on a constellation model that specifies the capabilities of each of the plurality of components;
determining, by the computer system, that the component should handle the request based on the capability of the component; and
routing, by the computer system, the request to the component.
10 Assignments
0 Petitions
Accused Products
Abstract
A system and method for processing multi-modal device interactions in a natural language voice services environment may be provided. In particular, one or more multi-modal device interactions may be received in a natural language voice services environment that includes one or more electronic devices. The multi-modal device interactions may include a non-voice interaction with at least one of the electronic devices or an application associated therewith, and may further include a natural language utterance relating to the non-voice interaction. Context relating to the non-voice interaction and the natural language utterance may be extracted and combined to determine an intent of the multi-modal device interaction, and a request may then be routed to one or more of the electronic devices based on the determined intent of the multi-modal device interaction.
-
Citations
33 Claims
-
1. A method for processing one or more multi-modal device interactions, received from a user, in a natural language voice services environment that includes a plurality of components that handle requests relating to the multi-modal device interact, the method being implemented on a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:
-
detecting, by the computer system, at least one multi-modal device interaction, wherein the multi-modal device interaction includes a non-voice interaction, from the user, with at least one of the plurality of components or an application associated with at least one of the plurality of components, and wherein the multi-modal device interaction further includes at least one natural language utterance, from the user, relating to the non-voice interaction; determining, by the computer system, a context relating to the non-voice interaction and a context relating to the natural language utterance; determining, by the computer system, an intent of the multi-modal device interaction based on the context relating to the non-voice interaction and the context of the natural language utterance; generating, by the computer system, a request based on the determined intent; obtaining, by the computer system, information indicating a capability of a component, from among the plurality of components, based on a constellation model that specifies the capabilities of each of the plurality of components; determining, by the computer system, that the component should handle the request based on the capability of the component; and routing, by the computer system, the request to the component. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A system for processing one or more multi-modal device interactions, received from a user, in a natural language voice services environment that includes a plurality of components that handle requests relating to the multi-modal device interact, comprising:
-
a computer system having one or more physical processors programmed with computer instructions that, when executed by the one or more physical processors, program the computer system to; detect at least one multi-modal device interaction, wherein the multi-modal device interaction includes a non-voice interaction, from the user, with at least one of the plurality of components or an application associated with at least one of the plurality of components, and wherein the multi-modal device interaction further includes at least one natural language utterance, from the user, relating to the non-voice interaction; determine a context relating to the non-voice interaction and a context relating to the natural language utterance; determine an intent of the multi-modal device interaction based on the context relating to the non-voice interaction and the context of the natural language utterance; generate a request based on the determined intent; obtain information indicating a capability of a component, from among the plurality of components, based on a constellation model that specifies the capabilities of each of the plurality of components; determine that the component should handle the request based on a capability of the component; and route the request to the component. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33)
-
Specification