Controlling an electronic component within an environment using a pointing device
First Claim
1. A computer-implemented process for controlling a user-selected electronic component within an environment using a pointing device, comprising:
- an inputting step for inputting an orientation message transmitted by the pointing device, said orientation message comprising orientation sensor readings generated by orientation sensors of the pointing device;
a recording step for recording a prescribed one or ones of the pointing device sensor outputs taken from the orientation message inputted whenever a user indicates the performance of a gesture representing a control action for the selected electronic component is taking place, the recording step comprising (a) a waiting step for waiting for the orientation message to be received;
(b) a switch state determining step for determining whether a switch state indicates that a pointing device switch has been activated by the user to indicate that the gesture is being performed, said switch state determining step being executed whenever the orientation message is received;
(c) a repeating step for repeating steps (a) and (b) whenever it is determined that the switch state does not indicate that the pointing device switch is activated;
(d) an output recording step for recording the prescribed one or ones of the pointing device sensor outputs taken from the received orientation message whenever it is determined that the switch state does indicate that the pointing device switch is activated, and;
a determining step for determining whether a threshold if just one, or all the thresholds if more than one, of a gesture threshold definition under consideration are exceeded by the recorded sensor output associated with the same sensor output as the threshold for the gesture threshold definition assigned to the selected electronic component;
a designating step for designating that the user has performed the gesture associated with the gesture threshold definition whenever it is determined that the threshold if just one, or all the thresholds if more than one, of the gesture threshold definition are exceeded by the recorded sensor output associated with the same sensor output; and
an implementing step for implementing a command represented by the gesture that the user was designated to have performed.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention is directed toward a system and process that controls a group of networked electronic components using a multimodal integration scheme in which inputs from a speech recognition subsystem, gesture recognition subsystem employing a wireless pointing device and pointing analysis subsystem also employing the pointing device, are combined to determine what component a user wants to control and what control action is desired. In this multimodal integration scheme, the desired action concerning an electronic component is decomposed into a command and a referent pair. The referent can be identified using the pointing device to identify the component by pointing at the component or an object associated with it, by using speech recognition, or both. The command may be specified by pressing a button on the pointing device, by a gesture performed with the pointing device, by a speech recognition event, or by any combination of these inputs.
-
Citations
2 Claims
-
1. A computer-implemented process for controlling a user-selected electronic component within an environment using a pointing device, comprising:
-
an inputting step for inputting an orientation message transmitted by the pointing device, said orientation message comprising orientation sensor readings generated by orientation sensors of the pointing device; a recording step for recording a prescribed one or ones of the pointing device sensor outputs taken from the orientation message inputted whenever a user indicates the performance of a gesture representing a control action for the selected electronic component is taking place, the recording step comprising (a) a waiting step for waiting for the orientation message to be received;
(b) a switch state determining step for determining whether a switch state indicates that a pointing device switch has been activated by the user to indicate that the gesture is being performed, said switch state determining step being executed whenever the orientation message is received;
(c) a repeating step for repeating steps (a) and (b) whenever it is determined that the switch state does not indicate that the pointing device switch is activated;
(d) an output recording step for recording the prescribed one or ones of the pointing device sensor outputs taken from the received orientation message whenever it is determined that the switch state does indicate that the pointing device switch is activated, and;a determining step for determining whether a threshold if just one, or all the thresholds if more than one, of a gesture threshold definition under consideration are exceeded by the recorded sensor output associated with the same sensor output as the threshold for the gesture threshold definition assigned to the selected electronic component; a designating step for designating that the user has performed the gesture associated with the gesture threshold definition whenever it is determined that the threshold if just one, or all the thresholds if more than one, of the gesture threshold definition are exceeded by the recorded sensor output associated with the same sensor output; and an implementing step for implementing a command represented by the gesture that the user was designated to have performed. - View Dependent Claims (2)
-
Specification