3D cursor or joystick device

• US 20020175897A1
• Filed: 05/31/2002
• Published: 11/28/2002
• Est. Priority Date: 08/27/1999
• Status: Active Grant

• Current Assignee: Maui Innovative Peripherals, Inc.
• Original Assignee: Maui Innovative Peripherals, Inc.
• Inventors: Pelosi, Michael J.
• Granted Patent: US 7,405,726 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G01S 3/784 using a mosaic of detectors
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 9/08 for teaching control of aircraft, e.g. Link trainer

Abstract

A 3D navigation system employs a complementary pair of emitter/detector units, one worn on a part of a user'"'"'s body (e.g., the head) and the other mounted in a stationary position on a display monitor facing the user. Each unit has an array of photovoltaic (PV) cells centered on a normal axis and an infrared diode for emitting a light beam on the normal axis. Each photodetectors array receives the infrared beam from the other unit and provides output signals representing the light intensities of the received beam on the PV cells. The PV cell response signals of the complementary units are used together to calculate the position and angular orientation of the user-worn unit relative to the stationary unit. The position and angle values are used by a software device driver installed on a computer to control 3D navigation functions of the associated application program on the computer. The system can be used for playing interctive 3D games, and also used for hands-free control in a wide range of other applications and environments, such as to control flight motions in a simulator or cockpit of an aircraft, or the command center of a naval vessel, spaceship, land vehicle, etc., for disabled persons to provide hands-free input to a computer, motorized wheelchair, prosthetic device, etc., and for controlling various functions in an industrial environment, such as guiding machine-assisted movement, or in CAD design, architectural, medical graphics, virtual reality, and other applications.

Claims

1. A 3D navigation system comprising:

a complementary pair of emitter/detector units, one of which is mounted on a movable input component moved by the user and the other of which is mounted in a stationary position facing toward the user, wherein each emitter/detector unit of the complementary pair has an emitter element for emitting a light beam centered on the unit'"'"'s normal axis in a direction toward the other unit, and an array of photodetectors arranged around the emitter element having their detection surfaces facing outwardly from the unit'"'"'s normal axis so as to receive the light beam from the other unit on its detection surfaces and provide an output of respective output signals representing light intensities of the beam received from the other unit on its respective detection surfaces, and a processing unit for receiving the output signals from the photodetector array of the movable unit and the output signals from the photodetector array of the stationary unit and using them as combined inputs to calculate position and angular orientation values representing the position and orientation angles of the movable unit relative to the stationary unit.

2. A 3D navigation system according to claim 1, wherein the movable unit is mounted on a part of the user'"'"'s body and controlled by moving that part of the body in 3D space in front of the stationary unit and without the requiring operation by the user'"'"'s hands.

3. A 3D navigation system according to claim 2, wherein the movable unit is mounted on the center of the user'"'"'s forehead, and the stationary unit is mounted on top of a display monitor in front of which the user is positioned.

4. A 3D navigation system according to claim 1, wherein the emitter/detector units of the complementary pair are of the same construction and operate with the same characteristics.

5. A 3D navigation system according to claim 4, wherein each emitter/detector units has four photovoltaic (PV) cells arranged in an X-configuration as the photodetector array and an infrared diode (IRED) as the emitter element in the center of the X-configuration.

6. A 3D navigation system according to claim 5, wherein each PV cell has a planar detection surface oriented at an angle of from 30 degrees to 45 degrees to the normal axis of the unit, and the IRED emits a beam of light that has a cone shape extending from 30 degrees to 45 degrees to the normal axis of the unit.

7. A 3D navigation system according to claim 5, wherein the PV cells are common silicon PV cells having a peak spectral response in the range of about 900 run, and the IRED is a GaAs diode having a peak spectral output in the range of about 900 nm.

8. A 3D navigation system according to claim 5, wherein the IREDs of the two units are pulsed On and Off in order to derive a difference signal is PV cell response values that removes the effect of background light.

9. A 3D navigation system according to claim 1, wherein each emitter/detector unit has an on-board operational amplifier for conversion of current response signals from the photodetector array to output voltage signals, and have their output voltage signals electrically coupled to a common processor module which converts the output voltage signals to binary device output values.

10. A 3D navigation system according to claim 9, wherein the processor module is connected to a communication port of an associated computer running an application program using the 3D navigation system, and the binary device output values are provided from the processor module to a software device driver installed on the computer for using the binary device output values as navigation control inputs for the application program.

11. A 3D navigation method comprising the steps of:

providing a complementary pair of emitter/detector units, one of which is mounted on a movable input component and the other of which is mounted in a stationary position facing toward the movable input component, wherein each emitter/detector unit has an emitter element for emitting a light beam toward the other unit, and a photodetector array for receiving the light beam from the other unit on its detection surfaces and providing output signals representing the light intensities of the beam received from the other unit on its respective detection surfaces; and

processing the output signals from the photodetector array of the movable unit and the output signals from the photodetector array of the stationary unit and using them as combined inputs to calculate position and angular orientation values representing the position and orientation angles of the movable unit relative to the stationary unit.

12. A 3D navigation method according to claim 11, wherein the movable unit is mounted on a part of the user'"'"'s body and controlled by moving that part of the body in 3D space in front of the stationary unit and without the requiring operation by the user'"'"'s hands.

13. A 3D navigation method according to claim 11, wherein the photodetector elements of the complementary units have response characteristics which vary in a predictable manner with the “

lamp-angle-of-incidence”

(LAOI) or “

offset”

effect due to a change in the relative orientation of the light beam of the other unit, with the “

angle of incidence”

(AOI) effect due to a change in the angle of incidence of the light beam on the detection surface of the photodetector element, and with the inverse-square-of-distance (Distance) effect due to a change in relative distance of the source of the light beam from the photodetector element.

14. A 3D navigation method according to claim 13, wherein the output signal from each photodetector element of the photodetector is given as follows:

Cell_Output=Distance_Effect * Angle_Incidence_Effect * Offset_Effect

15. A 3D navigation method according to claim 11, wherein the photodetector array of each unit (movable unit referred to as “

H”

unit, and stationary unit referred to as “

M”

unit) has four photodetector cells in identical X-configuration, referred to by numbers “

M1”

to “

M4” and

“

H1 to “

H4”

in clockwise direction, and the output signals of the four cells of each array are processed into Vertical Angle and Horizontal Angle values for the respective units, according to the following definition of their input values and solution equation used to solve them;

(a) M Vertical Angle Input Variables;

MVAx(1)=(M1−

M3)/((M1+M3)+0.000001)MVAx(2)=(H1−

H3)/((H1+H3)+0.000001)MVAx(3)=(M1−

M2)/((M1+M2)+0.000001)MVAx(4)=(M2−

M3)/((M2+M3)+0.000001)ACT_—

MVA=funcMVA(x)

(b) M Horizontal Angle Input Variables;

MHAx(1)=(M2−

M4)/((M2+M4)+0.000001)MHAx(2)=(H2−

H4)/((H2+H4)+0.000001)MHAx(3)=(M1−

M2)/((M1+M2)+0.000001)MHAx(4)=(M2−

M3)/((M2+M3)+0.000001)ACT_—

MHA=funcMHA(x)

(c) H Vertical Angle (to Normal Axis) Input Variables;

HVNAx(1)=(H1−

H3)/((H1+H3)+0.000001)HVNAx(2)=(M1−

M3)/((M1+M3)+0.000001)HVNAx(3)=(H2−

H3)/((H2+H3)+0.000001)HVNAx(4)=(H1−

H2)/((H1+H2)+0.000001)ACT_—

HVNA=funcHVNA(x)

(d) H Horizontal Angle (to Normal Axis) Input Variables;

HHNAx(1)=(H2−

H4)/((H2+H4)+0.000001)HHNAx(2)=(M2−

M4)/((M2+M4)+0.000001)HHNAx(3)=(H2−

H3)/((H2+H3)+0.000001)HHNAx(4)=(H1−

H2)/((H1+H2)+0.000001)ACT_—

HHNA=funcHHNA(x)

and wherein the above ACT_function is solved by curve-fitting to the following polynomial expansion equation;

$\begin{array}{c}\mathrm{ACT\_}<br><br>\left(\mathrm{Ax}\right)=\text{\hspace{1em}<br><br>}<br><br>a+b*X_1+c*X_2+d*X_1^2+e*X_2^2+f*X_1*X_2+g*X_1^3+\\ \text{\hspace{1em}<br><br>}<br><br>h*X_2^3+i*X_1*X_2^2+j*X_1^2*X_2+k+1*X_3+m*X_4+\\ \text{\hspace{1em}<br><br>}<br><br>n*X_3^2+o*X_4^2+p*X_3*X_4+q*X_3^3+r*X_4^3+s*X_3*\\ \text{\hspace{1em}<br><br>}<br><br>X_4^2+t*X_3^2<br><br>X_4.\end{array}$

16. A 3D navigation method according to claim 15, wherein the output signals of the four cells of the two units are processed into a Distance value DIST according to the following definition of input values and solution equation used to solve them:

DISTx(1)=SumHaoDISTx(2)=SumMaoDISTx(3)HLAOIeffectDISTx(4)=MLAOIeffectACT_—

DIST=funcDIST(x)

wherein these input factors are calculated as follows;

ACT_—

HLAOI=Atn(Sqr(Tan(ACT_—

MVA)^ 2+Tan(ACT_—

MHA)^

     2))ACT_—

MLAOI=Atn(Sqr(Tan(ACT_—

HVNA)^ 2+Tan(ACT_—

HHNA)^

     2))HLAOIeffect=LAOIa*ACT_—

HLAOI^ 8+LAOIb*ACT_—

HLAOI^ 7+LAOIc*ACT_—

HLAOI^ 6+LAOId*ACT_—

HLAOI^ 5+LAOIe*ACT_—

HLAOI^ 4+LAOIf*ACT_—

HLAOI^ 3+LAOIg*ACT_—

HLAOI^ 2+LAOIh*ACT_—

HLAOI+LAOIiH1AOI=Atn(Sqr(Tan((ACT_—

HVNA+0.52359))^ 2+Tan(ACT_—

HHNA)^

     2))H2AOI=Atn(Sqr(Tan((ACT_—

HVNA))^ 2+Tan((ACT_—

HHNA+0.52359))^

     2))H3AOI=Atn(Sqr(Tan((ACT_—

HVNA−

0.52359))^ 2+Tan(ACT_—

HHNA)^

     2))H4AOI=Atn(Sqr(Tan((ACT_—

HVNA))^ 2+Tan((ACT_—

HHNA−

0.52359))^

     2))H1AOIe=AOIa*H1AOI^ 5+AOIb*H1AOI^ 4+AOIc*H1AOI^ 3+AOId*H1AOI^ 2+AOIe*H1AOI+AOIfH2AOIe=AOIa*H2AOI^ 5+AOIb*H2AOI^ 4+AOIc*H2AOI^ 3+AOId*H2AOI^ 2+AOIe*H2AOI+AOIfH3AOIe=AOIa*H3AOI^ 5+AOIb*H3AOI^ 4+AOIc*H3AOI^ 3+AOId*H3AOI^ 2+AOIe*H3AOI+AOIfH4AOIe=AOIa*H4AOI^ 5+AOIb*H4AOI^ 4+AOIc*H4AOI^ 3+AOId*H4AOI^ 2+AOIe*H4AOI+AOIfH1ao=(H1/HLAOIeffect)/H1AOIeH2ao=(H2/HLAOIeffect)/H2AOIeH3ao=(H3/HLAOIeffect)/H3AOIeH4ao=(H4/HLAOIeffect)/H4AOIeSumHao=H1ao+H2ao+H3ao+H4aoMLAOIeffect=LAOIa*ACT_—

MLAOI^ 8+LAOIb*ACT_—

MLAOI^ 7+LAOIc*ACT_—

MLAOI^ 6+LAOId*ACT_—

MLAOI^ 5+LAOIe*ACT_—

MLAOI^ 4+_—

LAOIf*ACT_—

MLAOI^ 3+LAOIg*ACT_—

MLAOI^ 2+LAOIh*ACT_—

MLAOI+LAOIiM1AOI=Atn(Sqr(Tan((ACT_—

MVA+0.52359))^ 2+Tan(ACT_—

MHA)^

     2))M2AOI=Atn(Sqr(Tan((ACT_—

MVA))^ 2+Tan((ACT_—

MHA−

0.52359))^

     2))M3AOI=Atn(Sqr(Tan((ACT_—

MVA−

0.52359))^ 2+Tan(ACT_—

MHA)^

     2))M4AOI=Atn(Sqr(Tan((ACT_—

MVA))^ 2+Tan((ACT_—

MHA+0.52359))^

     2))M1AOIe=AOIa*M1AOI^ 5+AOIb*M1AOI^ 4+AOIc*M1AOI^ 3+AOId*M1AOI^ 2+AOIe*M1AOI+AOIfM2AOIe=AOIa*M2AOI^ 5+AOIb*M2AOI^ 4+AOIc*M2AOI^ 3+AOId*M2AOI^ 2+AOIe*M2AOI+AOIfM3AOIe=AOIa*M3AOI^ 5+AOIb*M3AOI^ 4+AOIc*M3AOI^ 3+AOId*M3AOI^ 2+AOIe*M3AOI+AOIfM4AOIe=AOIa*M4AOI^ 5+AOIb*M4AOI^ 4+AOIc*M4AOI^ 3+AOId*M4AOI^ 2+AOIe*M4AOI+AOIfM1ao=(M1/MLAOIeffect)/M1AOIeM2ao=(M2/MLAOIeffect)/M2AOIeM3ao=(M3/MLAOIeffect)/M3AOIeM4ao=(M4/MLAOIeffect)/M4AOIeSumMao=M1ao+M2ao+M3ao+M4ao

and wherein the above ACT_function is solved by curve-fitting to the following polynomial expansion equation;

18. A 3D navigation solution method according to claim 17, wherein the processing of output signals of the photodetector arrays includes:

(i) building a software model or simulation of the “

lamp-angle-of-incidence”

(LAOI) effect, “

angle-of-incidence”

(AOI) effect, and “

inverse-square-of-distance”

(Distance) effects of the response output signals of the light beam of one unit on the photodetector array of the other unit; and

(ii) finding a “

best fitting”

polynomial solution equation by curve-fitting the modeled response outputs to a calibration set of raw cell signal outputs to derive a “

best fit”

solution set of polynomial coefficients.

19. A 3D navigation solution method according to claim 18, wherein the processing of output signals of the photodetector arrays includes:

(iii) dividing a solution space over which the photodetector arrays are operative into a plurality of solution segments;

(iv) solving for “

best fit”

solution sets of polynomial coefficients for each of the segments of the solution space; and

(v) using the solution set of polynomial coefficients for each solution segment, respectively, by applying the coefficient set to the curve-fitting polynomial for solving for desired values in the respective segment of the solution space.

20. A 3D navigation solution method according to claim 19, wherein each solution segment is addressed by segmentation of the solution space into vertical angle, horizontal angle, and distance segments of the movable unit relative to the stationary unit.

17. A 3D navigation solution method comprising the steps of:

providing a complementary pair of emitter/detector units, one of which is mounted on a movable input component and the other of which is mounted in a stationary position facing toward the movable input component, wherein each emitter/detector unit has an emitter element for emitting a light beam toward the other unit, and a photodetector array for receiving the light beam from the other unit on its detection surfaces and providing output signals representing the light intensities of the beam received from the other unit on its respective detection surfaces; and

processing the output signals from the photodetector array of the movable unit and the output signals from the photodetector array of the stationary unit and using them as combined inputs to calculate position and angular orientation values representing the position and orientation angles of the movable unit relative to the stationary unit.