Direct navigation of two-dimensional control using a three-dimensional pointing device

• US 7,859,523 B2
• Filed: 08/10/2004
• Issued: 12/28/2010
• Est. Priority Date: 08/10/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kong, Yuan
• Primary Examiner(s): Nguyen; Chanh
• Assistant Examiner(s): WALTHALL, ALLISON N
• Application Number:
  US10/914,118
• Publication Number:
  US 20060033711A1
• Time in Patent Office: 2,331 Days
• Field of Search: 345156-159, 345/179, 345/163, 345/166, 715/754, 715856-858, 715/863, 463/37
• US Class Current: 345/179
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Direct and absolute pointing is provided for with respect to a two-dimensional information display surface, much like how one would point a laser pointer or flashlight at a desired point. The displayed control may be moved by manipulating the pointing device in three dimensions. The translational position of the pointing device may be measured in three dimensions. Also, the three-dimensional orientation of the pointing device may be measured. A computing device may receive this information from the pointing device and determine where the pointing device is pointing to. If the pointing device is pointing at a display, then the computing device may cause the control to be displayed at the position to which the pointing device is pointing. In addition, the control may be displayed at an orientation that depends upon the orientation of the pointing device.

Claims

1. In a computing device, a method for providing absolute positioning of a displayed control, comprising steps of:

calibrating, in an X-Y-Z coordinate system, a pointing device in at least five degrees of freedom using a plurality of known reference locations, wherein the calibration allows the pointing device to accurately control the displayed control based on a translational position and an orientation of the pointing device;

the calibrating of the pointing device is comprised of;

(1) defining an X position, a Y position, an orientation about an X axis, and an orientation about a Y axis equal to zero as the pointing device is pointed to a first location of the plurality of known reference locations;

(2) defining one of the following as equal to zero as the pointing device is pointed to a second location of the plurality of known reference locations;

a) a second X position and a second orientation about the Y axis, or

b) a second Y position and a second orientation about the X axis; and

(3) calculating a distance from the pointing device to a geometric plane in which the first location and the second location coexist;

determining the translational position of the pointing device along first, second, and third degrees of freedom, wherein the translational position is determined utilizing inertial tracking technology;

determining the orientation of the pointing device along fourth and fifth degrees of freedom, wherein the orientation is determined utilizing optical tracking technology;

controlling a display device to display the displayed control at a control position that depends upon the translational position of the pointing device along the first, second, and third degrees of freedom and that further depends upon the orientation of the pointing device along the fourth and fifth degrees of freedom; and

controlling the display device to display the displayed control at the control position with an orientation that corresponds with an angle of orientation of the pointing device around a pointing axis of the pointing device.

2. The method of claim 1, further including a step of controlling the display device to display an element at each of the plurality of known reference locations.

3. The method of claim 1, wherein the plurality of known reference locations are locations separate from the display.

4. The method of claim 1, wherein the displayed control is a mouse pointer.

5. The method of claim 1, wherein the display device includes a plurality of display screens.

6. The method of claim 1, wherein the pointing device has a body, a linear axis being defined that is fixed with respect to the body and parallel to the pointing axis, and wherein the step of controlling a display device to display the displayed control includes determining a projected location on the display device to which the linear axis projects and defining the control position of the displayed control to be the projected location.

7. In a computing device, a method for providing absolute positioning of a displayed control, comprising steps of:

controlling a display device to display an element at each of a plurality of known reference locations, wherein the display device is associated with a two-dimensional coordinate system;

calibrating at least five degrees of freedom of a pointing device in a three-dimensional X-Y-Z coordinate system environment using the plurality of known reference locations, wherein the calibration of the pointing device is comprised of;

(1) defining an X position, a Y position, an orientation about an X axis, and an orientation about a Y axis equal to zero as the pointing device is pointed to a first location of the plurality of known reference locations;

(2) defining one of the following as equal to zero as the pointing device is pointed to a second location of the plurality of known reference locations;

a) a second X position and a second orientation about the Y axis, or

b) a second Y position and a second orientation about the X axis; and

(3) calculating a distance from the pointing device to a geometric plane in which the first location and the second location coexist;

receiving a first signal representing a first translational position and a first orientation of the pointing device along at least five degrees of freedom;

determining a first control position of the displayed control that depends upon the first translational position and the first orientation of the pointing device along the at least five degrees of freedom, wherein the first control position of the displayed control is determined by mapping the first translational position and the first orientation of the pointing device to the two-dimensional coordinate system;

displaying the displayed control on the display device at the first control position;

displaying the displayed control on the display device at the first control position with an orientation that corresponds with an angle of orientation of the pointing device around a pointing axis of the pointing device;

receiving a second signal representing a second translational position and a second orientation of the pointing device along the at least five degrees of freedom;

determining a second control position of the displayed control that depends upon the second translational position and the second orientation of the pointing device along the at least five degrees of freedom, wherein the second control position of the displayed control is determined by mapping the second translational position and the second orientation to the two-dimensional coordinate system; and

displaying the displayed control on the display device at the second control position.

8. The method of claim 7, wherein the pointing device defines a linear axis fixed with respect to the pointing device that is parallel to the pointing axis, the step of determining the first control position of the displayed control further includes determining a projected location of the linear axis outside of the pointing device and determining the first control position of the control depending upon the projected location.

9. In a computing device, a method for providing absolute positioning of a displayed control, comprising steps of:

calibrating a pointing device using a plurality of known reference locations, wherein the calibration allows the pointing device to accurately control the displayed control based on a translational position and an orientation of the pointing device in a three-dimensional X-Y-Z coordinate system environment, the calibrating of the pointing device comprising;

(1) defining an X position, a Y position, an orientation about an X axis, and an orientation about a Y axis equal to zero as the pointing device is pointed to a first location of the plurality of known reference locations;

(2) defining one of the following as equal to zero as the pointing device is pointed to a second location of the plurality of known reference locations;

a) a second X position and a second orientation about the Y axis, or

b) a second Y position and a second orientation about the X axis; and

(3) calculating a distance from the pointing device to a geometric plane in which the first location and the second location coexist;

(4) mapping the translational position and the orientation of the pointing device at each of the first location and the second location in the three-dimensional environment to a two-dimensional coordinate system of a display;

determining a first translational position of the pointing device in the three-dimensional environment, wherein the first translational position is determined utilizing a first tracking technology;

determining a first orientation of the pointing device in the three-dimensional environment, wherein the first orientation of the pointing device is determined utilizing a second tracking technology, and wherein the first tracking technology and the second tracking technology are different tracking technologies;

controlling the display to display the displayed control at a control position that depends upon the first translational position and the first orientation of the pointing device in the three dimensions; and

controlling the display to display the displayed control at the control position with an orientation that corresponds with an angle of orientation of the pointing device around a pointing axis of the pointing device.

10. An apparatus for use with a computing device, comprising:

a body;

a first sensor configured to determine a translational position of the body;

a second sensor configured to determine an orientation of the body, wherein the first sensor and the second sensor determine the translational position and the orientation of the body along at least five degrees of freedom, wherein the first sensor utilizes one of optical tracking technology or inertial tracking technology, and wherein the second sensor utilizes the other of optical tracking technology or inertial tracking technology; and

a computer-readable medium storing computer-executable instructions configured, when executed by a processor unit, to cause the computer to perform steps of;

calibrating in at least five degrees of freedom the first sensor and the second sensor using a plurality of known reference locations, wherein the calibration allows the first sensor and the second sensor to accurately control a displayed control based on the translational position and the orientation of the body;

the calibrating in at least five degrees is comprised of;

(1) defining an X position, a Y position, an orientation about an X axis, and an orientation about a Y axis equal to zero as the pointing device is pointed to a first location of the plurality of known reference locations;

(2) defining one of the following as equal to zero as the pointing device is pointed to a second location of the plurality of known reference locations;

a) a second X position and a second orientation about the Y axis, or

b) a second Y position and a second orientation about the X axis; and

(3) calculating a distance from the pointing device to a geometric plane in which the first location and the second location coexist;

receiving a signal periodically from the first sensor representing the translational position of the body and receiving a signal periodically from the second sensor representing the orientation of the body, and

determining a control position of a control that depends upon the translational position and the orientation of the body along the at least five degrees of freedom.

displaying the control on a display at the control position, the control having an orientation as displayed that corresponds with an angle of orientation of the pointing device around a pointing axis of the pointing device.

11. The apparatus of claim 10, wherein the second sensor is physically coupled to the body.

12. The apparatus of claim 10, wherein the body has first and second ends defining a linear axis passing through the first and second ends that is parallel to the pointing axis, the step of determining the control position of the control including determining a projected location of the linear axis outside of the body and determining the control position of the control depending upon the projected location.

13. The apparatus of claim 12, wherein the second sensor utilizes optical tracking technology.

14. The apparatus of claim 13, wherein the optical tracking technology utilizes one or more cameras.

15. The apparatus of claim 10 further comprising a communication connection for receiving the signal from the first sensor.

16. The apparatus of claim 10, wherein the computer-executable instructions are further configured, when executed by a processor unit, to cause the computing device to perform a step of displaying on a display device the control at the control position.