System and process for selecting objects in a ubiquitous computing environment
First Claim
1. A computer-implemented process for selecting an object within an environment by a user pointing to the object with a pointing device, comprising:
- an inputting step for inputting orientation sensor readings generated by orientation sensors of the pointing device and provided by the device in an orientation message;
a deriving step for deriving the orientation of the pointing device in relation to a predefined coordinate system of the environment from the inputted orientation sensor readings;
a second inputting step for inputting digital video from a pair of video cameras each of which is located so as to capture images of the environment from different viewpoints;
an ascertaining step for ascertaining the location of the pointing device at a time substantially contemporaneous with the generation of the orientation message and in terms of the predefined coordinate system using the inputted digital video from the pair of video cameras;
a determining step for using the orientation and location of the pointing device to determine whether the pointing device is being pointed at an object in the environment that is controllable by the computer; and
whenever the pointing device is being pointed at a controllable object, a selecting step for selecting that object for future control actions.
1 Assignment
0 Petitions
Accused Products
Abstract
A system and process for selecting objects in an ubiquitous computing environment where various electronic devices are controlled by a computer via a network connection and the objects are selected by a user pointing to them with a wireless RF pointer. By a combination of electronic sensors onboard the pointer and external calibrated cameras, a host computer equipped with an RF transceiver decodes the orientation sensor values transmitted to it by the pointer and computes the orientation and 3D position of the pointer. This information, along with a model defining the locations of each object in the environment that is associated with a controllable electronic component, is used to determine what object a user is pointing at so as to select that object for further control actions.
-
Citations
8 Claims
-
1. A computer-implemented process for selecting an object within an environment by a user pointing to the object with a pointing device, comprising:
-
an inputting step for inputting orientation sensor readings generated by orientation sensors of the pointing device and provided by the device in an orientation message; a deriving step for deriving the orientation of the pointing device in relation to a predefined coordinate system of the environment from the inputted orientation sensor readings; a second inputting step for inputting digital video from a pair of video cameras each of which is located so as to capture images of the environment from different viewpoints; an ascertaining step for ascertaining the location of the pointing device at a time substantially contemporaneous with the generation of the orientation message and in terms of the predefined coordinate system using the inputted digital video from the pair of video cameras; a determining step for using the orientation and location of the pointing device to determine whether the pointing device is being pointed at an object in the environment that is controllable by the computer; and whenever the pointing device is being pointed at a controllable object, a selecting step for selecting that object for future control actions. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A computer-readable medium having computer-executable instructions for selecting an object within an environment by a user pointing to the object with a pointing device, said computer-readable instructions comprising:
-
inputting orientation sensor readings generated by orientation sensors of the pointing device and provided by the device in an orientation message; deriving the orientation of the pointing device in relation to a predefined coordinate system of the environment from the inputted orientation sensor readings; inputting digital video from a pair of video cameras each of which is located so as to capture images of the environment from different viewpoints; ascertaining the location of the pointing device at a time substantially contemporaneous with the generation of the orientation message and in terms of the predefined coordinate system using the inputted digital video from the pair of video cameras; using the orientation and location of the pointing device to determine whether the pointing device is being pointed at an object in the environment that is controllable by the computer; and whenever the pointing device is being pointed at a controllable object, selecting that object for future control actions.
-
Specification