System and process for controlling electronic components in a ubiquitous computing environment using multimodal integration
First Claim
1. A multimodal system for controlling electronic components, comprising:
- a general purpose computing system which is in communication with said electronic components via a computer network, said electronic components being separate from the computing system;
a computer program comprising program modules executable by the computing system, said program modules comprising an object selection module, a gesture recognition module, and a speech control module, each of which provides inputs to an integration module that integrates said inputs to arrive at a unified interpretation of what component a user wants to control and what control action is desired.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
45 Citations
27 Claims
-
1. A multimodal system for controlling electronic components, comprising:
-
a general purpose computing system which is in communication with said electronic components via a computer network, said electronic components being separate from the computing system;
a computer program comprising program modules executable by the computing system, said program modules comprising an object selection module, a gesture recognition module, and a speech control module, each of which provides inputs to an integration module that integrates said inputs to arrive at a unified interpretation of what component a user wants to control and what control action is desired. - View Dependent Claims (2)
-
-
3-11. -11. (canceled)
- 12. A computer-implemented multimodal electronic component control process comprising a pointer-based object selection process module, a gesture recognition process module, and a speech control process module, each of which provides inputs to an integration process module that integrates the inputs to arrive at a unified interpretation of what component a user wants to control and what control action is desired, wherein the electronic component being controlled is separate from a computer system implementing the multimodal electronic component control process and is in communication with the computer system via a computer network.
-
15-20. -20. (canceled)
-
21. A computer-readable medium having computer-executable instructions for causing a computer system to control electronic components using multimodal integration, said computer-executable instructions comprising:
-
accepting inputs from at least one of a pointer-based object selection process module, a gesture recognition process module, and a speech control process module; and
integrating said inputs to arrive at a unified interpretation of what electronic component a user wants to control and what control action is desired, wherein the electronic components being controlled are separate from the computer system but are in communication with the computer system via a computer network. - View Dependent Claims (22, 23, 24, 25, 26)
-
-
27-57. -57. (canceled)
Specification