Three-dimensional virtual input and simulation apparatus

• US 8,368,647 B2
• Filed: 08/01/2008
• Issued: 02/05/2013
• Est. Priority Date: 08/01/2007
• Status: Expired due to Fees

• Current Assignee: Ming-Yen Lin
• Original Assignee: Ming-Yen Lin
• Inventors: Lin, Ming-Yen
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): XAVIER, ANTONIO J
• Application Number:
  US12/184,475
• Publication Number:
  US 20090033623A1
• Time in Patent Office: 1,649 Days
• Field of Search: 345156-184
• US Class Current: 345/158
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

The present invention relates to a three-dimensional virtual input and simulation apparatus, and more particularly to an apparatus comprising a plurality of point light sources, a plurality of optical positioning devices with a visual axis tracking function, and a control analysis procedure. The invention is characterized in that the plurality of optical positioning devices with the visual axis tracking function are provided for measuring and analyzing 3D movements of the plurality of point light sources to achieve the effect of a virtual input and simulator.

Claims

1. A three-dimensional virtual input and simulation apparatus, comprising:

a plurality of point light sources, wherein each point light source is capable of simultaneously and continuously emitting a point light source which is scattered;

a plurality of 1D optical positioning devices, wherein each 1D optical positioning device having a visual axis tracking function receives a synchronization enable signal and all scattered light sources simultaneously emitted by the point light sources for measuring the 3D positions of all point light sources and outputting a set of physical quantities, and wherein each 1D optical positioning devices is equipped with a visual axis tracking function for automatically tracking the central group coordinate of all point light sources and the coordinate of one of the point light sources and outputting a visual axis angle of its own visual axis so as to achieve the visual axis tracking purpose, and each 1D optical positioning devices further is equipped with a visual axis positioning function to receive a visual axis angle for setting the angular direction of its own visual axis, so as to achieve the visual axis positioning purpose; and

a control analysis unit, having a control analysis procedure being a software procedure, mainly used to connect and control all 1D optical positioning devices, wherein the control analysis unit outputs a synchronization enable signal for synchronously starting all 1D optical positioning devices to do the 3D position measurement, also outputs a set of visual axis angles for setting the angular direction of visual axis of each 1D optical positioning devices, so as to achieve the visual axis positioning purpose for all 1D optical positioning devices, and further receives all physical quantities and a set of a visual axis angles for simulating an input function of a physical input device, so as to achieve the virtual input purpose, and also simulates the motion of a physical object, so as to achieve the purpose of simulator;

wherein the single 1D optical positioning devices with a visual axis tracking function comprises;

a plurality of linear positioning sensors, wherein each linear positioning sensor receives a synchronization scan signal and simultaneously receives all scattering light sources emitted by the point light sources for calculating and outputting the image mean positions of all point light sources;

a positioning calculation control microprocessor, having a positioning calculation and control procedure, connected to control all linear positioning sensors and the two-axis angular control device and mainly receiving the synchronization enable signal, all image mean positions, a visual axis angle and two feedback angular electric signals so as to calculate and output a synchronization scan signal, a set of physical quantities, a visual axis angle and two angular driving signals;

a signal transmission interface, being a wire or a wireless transmitting device, for transmitting a set of physical quantities, a visual axis angle and a synchronization enable signal;

a group of pilot point light sources, being composed of a plurality of point light sources and installed and fixed at known positions of the 1D optical positioning device fixation mechanism, for being used for the position measurement and the visual axis angle positioning of the 1D optical positioning device;

a 1D optical positioning device fixation mechanism, being a mechanical structure, for installing and fixing the plurality of linear positioning sensors, the positioning calculation control microprocessor, the signal transmission interface and the group of pilot point light sources, and connecting to the two-axis rotation mechanism in the two-axis angular control device to achieve a two-axis rotation effect; and

a two-axis angular control device, being composed of two actuators, two angular sensors and a two-axis rotation mechanism, for receiving two angular driving signals, whereby driving the two actuators to rotate the two-axis rotation mechanism and the two angular sensors according to the magnitude of the angular driving signal, the two angular sensors outputting two angular electric signals as feedback signals according to the magnitude of the actual rotation, so as to achieve the angular positioning effect for the two-axis rotation mechanism, and the two-axis rotation mechanism rotating the 1D optical positioning device fixation mechanism, so as to change the visual axis angle of the 1D optical positioning device.

2. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the each point light source has a unique characteristic of light intensity, and preferably has a light emission with an equal light emitting radius but a different light emitting intensity.

3. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the each point light source has a unique characteristic of geometric size, and preferably has a light emission with a different light emitting radius but an equal light emitting intensity.

4. Three-dimensional virtual input and simulation apparatus of claim 1, wherein each point light source in the plurality of point light sources has a unique characteristic of wavelength, preferably has a light emission with a different light emitting wavelength, and each wavelength is not overlapped.

5. Three-dimensional virtual input and simulation apparatus of claim 4, wherein the number of the plurality of point light sources is three, and the light emitting wavelength of each point light source are the red, green and blue light.

6. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the composed unique characteristics of the plurality of point light sources is a combination of light intensity, geometric size and wavelength.

7. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the single point light source comprises:

a light scattering component, being an object capable of uniformly scattering an incident light along all angular directions;

a light emitting source, being composed of one or multiple LED or semiconductor laser diodes, for emitting the visible light or non-visible light;

an electronic control circuit, including a power switch and a circuit of a constant current source, for providing the function of a power switch and a constant current source which is supplied to the light emitting source for obtaining a specific and stable brightness of the light emission;

a battery, for supplying an electric power to the light emitting source and the electronic control circuit;

a point light source fixation mechanism, being a mechanical mechanism, for installing and fixing the light scattering component, the light emitting source, the electronic control circuit and the battery; and

a device fixation mechanism, being a mechanical mechanism, for attaching and fixing the point light source fixation mechanism onto an object.

8. Three-dimensional virtual input and simulation apparatus of claim 7, wherein the light scattering component is comprised of a transparent optical guide material and a scattering material, the transparent optical guide material is a geometric shape, and is made of transparent material, the scattering material is installed inside the transparent optical guide material and the scattering material is a randomly distributed powder with a light reflecting capability, or a randomly distributed transparent powder, or a randomly distributed tiny air bubbles, or a smaller and transparent ball, further, a position located near the transparent optical guide material has a light incident window with a predefined size and shape for allowing the incident light emitted from the light emitting source to enter the transparent optical guide material through a angle.

9. Three-dimensional virtual input and simulation apparatus of claim 7, wherein the light emitting source further uses an optical bandpass filter to generate a light source with a specific wavelength.

10. Three-dimensional virtual input and simulation apparatus of claim 7, wherein the device fixation mechanism is attached to one of a set of the fingers of a hand, a forehead of a head, an instep of a foot, a racket-shaped object, a rod-shaped object, a stick-shaped object, a glove-shaped object, a spherical object, a toy, a remote control toy, a joystick, and a remote control of a home video game player.

11. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the single linear positioning sensor comprises:

a 1D optical element, comprising a filter, a linear aperture, and 1D optical lens;

a linear optical sensor, comprising a linear optical sensor array, an electronic scan and read circuit and an analog-to-digital converter, the electronic scan and read circuit sequentially and continuously reading and outputting an analog voltage sensed by each photosensitive pixel of the linear optical sensor array according to the received scanning timing signal, the analog-to-digital converter converting the analog voltage and outputting a digital voltage which is the image superposed signal;

a signal processing microprocessor, being connected to control the linear optical sensor, for receiving the synchronization scan signal to start the execution of a signal processing procedure which outputs a scanning timing signal and reads the image superposed signal, so as to calculate and output the image mean position of all point light sources; and

a linear positioning sensor fixation mechanism, being a mechanical structure, for installing and fixing the 1D optical element, the linear optical sensor and the signal processing microprocessor, and being installed and fixed in the 1D optical positioning device fixation mechanism.

12. Three-dimensional virtual input and simulation apparatus of claim 11, wherein the linear optical sensor is composed of a linear color optical sensor array, an electronic scan and read circuit and an analog-to-digital converter, each photosensitive pixel of the linear color optical sensor array installs a proper color filter, and the preferable color filters are R, G, and B color filters.

13. Three-dimensional virtual input and simulation apparatus of claim 11, wherein the linear optical sensor comprises a plurality of linear color optical sensor array, an electronic scan and read circuit and an analog-to-digital converter, all photosensitive pixels in each linear color optical sensor array install a proper color filter, and the preferable color filters are R, G, and B color filters.

14. Three-dimensional virtual input and simulation apparatus of claim 11, wherein the linear optical sensor comprises a 2D color optical sensor array, an electronic random access circuit and an analog-to-digital converter, the 2D color optical sensor array comprises a plurality of linear color optical sensor arrays, the color filters are arranged in a way that each photosensitive pixel of each linear color optical sensor array installs a proper color filter, preferably R, G, and B color filters, or all photosensitive pixels in each linear color optical sensor array install a proper color filter, preferably R, G, and B color filters, the electronic random access circuit comprises a microprocessor, a row decoder and a column decoder for reading a pixel directly and randomly.

15. Three-dimensional virtual input and simulation apparatus of claim 11, wherein the signal processing procedure comprises:

a synchronous data acquisition procedure, for receiving the synchronization scan signal, whereby outputting a scanning timing signal at a proper time, so as to obtain and record the image superposed signal which includes the effective image signal formed by all point light sources and the dynamic background light signal;

a dynamic background light signal elimination procedure, comprising an elimination procedure of time dependent ambient light interference signal and an elimination procedure of space dependent ambient light interference signal, and being able to eliminate the dynamic background light signal from the image superposed signal, so as to output an effective image signal of all point light sources;

an image signal recognition and correspondence procedure, mainly utilizing a threshold comparison and/or a profile detection method to analyze and recognize the effective image signal of all point light sources and obtain the correspondence relationship, the profile detection method using the characteristics of the effective image signal, such as the standard deviation, the central intensity and the slope of the waveform, to achieve the purpose of analysis, recognition and correspondence, further, when adopting the geometric modulation method, a method of deduction and a Gaussian Fitting being used to analyze and recognize the effective image signal of all point light sources and obtain the correspondence relationship; and

an image mean position calculation procedure, for applying several analytic processes such as the analysis of the position of the brightest sensed pixel, the analysis of Gaussian Fitting and the analysis of statistics to each analyzed and recognized effective image signal for calculating and outputting each image mean position.

16. Three-dimensional virtual input and simulation apparatus of claim 15, wherein the dynamic background light signal elimination procedure is a hardware method which uses one more noise measurement 1D optical sensor, by utilizing the same scanning timing used by the 1D optical sensor and a proper signal amplification, the noise measurement 1D optical sensor obtains the dynamic background light signal independently, thus the dynamic background light signal is subtracted from the image superposed signal, so as to achieve the purpose of the elimination of the dynamic background light signal, a proper optical filter attached to the linear optical sensor array in the noise measurement linear optical sensor is used to block out the light sources emitted by the point light sources but transmit the background light sources.

17. Three-dimensional virtual input and simulation apparatus of claim 15, wherein the elimination procedure of time dependent ambient light interference signal is a software method which applies a mathematic subtraction process to the two consecutively obtained image superposed signals, so as to achieve the purpose of eliminating the dynamic background light signal.

18. Three-dimensional virtual input and simulation apparatus of claim 15, wherein the elimination procedure of space dependent ambient light interference signal is a Fourier signal process, the Fourier signal process first applies a Fourier transformation to the data obtained by the elimination procedure of time dependent ambient light interference signal, then applies a bandpass filtering process to the transformed data for reducing the unnecessary frequencies and amplifying the signal in the frequency domain, and finally applies an inverse Fourier transformation to the processed data in the frequency domain, so as to achieve the purpose of eliminating the space dependent ambient light interference.

19. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the positioning calculation and control procedure in the positioning calculation control microprocessor comprises:

a synchronous scan procedure, for receiving a synchronization enable signal to periodically generate and output a synchronization scan signal, so as to synchronously drive all linear positioning sensors to start the execution of a signal processing procedure;

a physical quantity calculation procedure, for obtaining the image mean positions provided by the linear positioning sensors and calculating and outputting a set of physical quantities, the set of physical quantities including the physical quantity, the group physical quantity, the relative physical quantity, and other physical quantities; and

a visual axis control procedure, for calculating and outputting a visual axis angle and two angular driving signals according to the physical quantity, group physical quantity or a visual axis angle, further receiving two feedback angular electric signals to implement an angular feedback control, so as to achieve the angular positioning control of the visual axis.

20. Three-dimensional virtual input and simulation apparatus of claim 19, wherein the physical quantity means the physical quantities comprising the 3D position coordinates, 3D displacements, 3D velocities and 3D accelerations of the point light sources, and the group physical quantity means the physical quantities comprising a group center coordinate, a group average displacement, a group average velocity, and a group average acceleration, and the relative physical quantity means the physical quantities comprising a relative position, a relative velocity, a relative acceleration, an angle, an angular velocity, an angular acceleration and a plan vector constructed among the point light sources and the group center coordinates, and other physical quantities means the physical quantities comprising the force, torque, centripetal force, momentum and kinetic energy of the point light sources.

21. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the 1D optical positioning device fixation mechanism is a triangular geometric structure, preferably an equilateral triangle structure, and installs and fixes a the linear positioning sensor fixation mechanism at each vertices or the middle of three sides of the triangular device, namely the relative installed positions of three linear positioning sensors constitute a triangular geometric structure, further the linear positioning sensor can be set to any angle by the rotation about its optical axis, namely the direction of a long axis of the linear optical sensor array in the linear positioning sensors can be set to any angle, and the group of pilot point light sources are installed at any position of the triangular 1D optical positioning device fixation mechanism, and the preferable number of the pilot point light sources is three, and the preferable installed positions are the vertices or the middle of three sides of the triangular, further a connecting structure can be installed at the vertices of the triangular 1D optical positioning device fixation mechanism, and the connecting structure includes a structure which can connects or detaches two sides of a triangle and can freely adjust the angle included between any two sides of the triangle.

22. Three-dimensional virtual input and simulation apparatus of claim 21, wherein the 1D optical positioning device fixation mechanism installs one more connecting mechanism installed at the middle of any one of three sides such that an extra linear positioning sensor is installed, and the extra linear positioning sensor is installed at the center of the triangle.

23. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the 1D optical positioning device fixation mechanism is the improved mechanism of claim 21, has a quadrilateral, a pentagonal or a hexagonal geometric structure, preferably a square, an equilateral pentagon or an equilateral hexagon, and installs and fixes a linear positioning sensor fixation mechanism at each vertices or the middle of each sides of the 1D optical positioning device fixation mechanism, namely the number of installed linear positioning sensor is four, five or six, further the group of pilot point light sources are installed at any position of the 1D optical positioning device fixation mechanism, and the preferable number of the pilot point light sources is three, and the preferable installed position are the vertices or the middle of each sides of the 1D optical positioning device fixation mechanism.

24. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the 1D optical positioning device fixation mechanism can be a casing of other conventional devices such as a notebook computer, a PDA, a video game player, a mobile phone, a liquid crystal display, a plasma display, a television, a projector, an optical camera, an optical camcorder, an optical telescope, an automobile or a motorcycle, namely the plurality of linear positioning sensors, the positioning calculation control microprocessor, the signal transmission interface and the pilot point light sources can be installed on the casing of the aforementioned conventional devices.

25. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the control analysis procedure in the control analysis unit further comprises:

a coordinate alignment and synchronous calibration procedure, for obtaining the coordinate transformation relationship between any two of 1D optical positioning devices with a visual axis tracking function, and compensating the positioning error caused by the coordinate transformation, and correcting the error of synchronous timing;

a device simulated input procedure, for recognizing and simulating the operating gestures required by a physical input device, as to achieve the virtual input purpose; and

a simulator procedure, for measuring the positions of a plurality of point light sources installed on a physical object in real time to obtain the trajectory and physical movement quantities of the physical object, and further taking the advantage of the virtual reality technology to define a virtual object in the virtual space and corresponding the motion of the virtual object to the motion of the physical object, and thus the virtual object interacting with other virtual objects in the virtual space according to the physical laws, so as to achieve the purpose of simulation.

26. Three-dimensional virtual input and simulation apparatus of claim 25, wherein the coordinate alignment and synchronous calibration procedure comprises:

a visual axis reset procedure, for using the visual axis control procedure installed in the 1D optical positioning device with a visual axis tracking function to point the visual axes of 1D optical positioning devices all at a positioning point light source and thus setting the angle of each visual axis at zero degree;

a coordinate transformation setting procedure, for selecting one of the 1D optical positioning device as a master positioning device and the others as client positioning devices, the master positioning device measuring the position of the positioning point light source and each pilot point light sources of all client positioning devices and each client positioning device measuring the position of the positioning point light source, such that the coordinate transformation relationship between the master and the client positioning devices and the compensation of the positioning error being obtained; and

a synchronous timing calibration procedure, for periodically outputting a synchronization enable signal in a proper cycle of time interval, such that all positioning device being calibrated to execute the positioning calculation and control procedure synchronously.

27. Three-dimensional virtual input and simulation apparatus of claim 25, wherein the device simulated input procedure comprises:

a procedure of virtual operating screen correspondence, as to a physical operating screen with an actual dimension, for defining a virtual operating screen at any position in the space, the virtual operating screen having a spatial corresponding relationship with the physical operating screen, and the relationship being a one-to-one corresponding relation which may have a magnifying corresponding relation, an equivalent corresponding relation or a shrinking corresponding relation, and further by taking the advantage of virtual reality technology, the virtual operating screen being visually displayed as a virtual stereo image;

a procedure of virtual device geometric structure definition and operating fingers correspondence, as to a physical input device to be simulated, for defining a geometric structure of a virtual device, a physical position and size of the virtual function keys and the physical motion of the virtual function keys, and corresponding the operating fingers to the virtual function keys, and further by taking the advantage of virtual reality technology, the geometric structure of the virtual device and operating fingers being visually displayed as a virtual stereo image; and

a procedure of operating gestures definition and recognition, according to the physical motion of the virtual function keys, for defining the physical movement quantities of the operating fingers, the physical movement quantities being a set of consecutive time dependent physical quantities, and the set of physical quantities comprising the physical quantities, group physical quantities, relative physical quantities and other physical quantities of all point light sources, thus according to these predefined physical quantities, the gestures of operating fingers being recognized by measuring and analyzing the motion of point light sources installed on the fingers, so as to achieve the device simulated input purpose.

28. Three-dimensional virtual input and simulation apparatus of claim 27, wherein the physical operating screen is the Windows operating screen of a personal computer, the Windows operating screen of a PDA, the operating screen of a mobile phone, the operating screen of a television or the operating screen of a game player.

29. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the control analysis procedure is integrated and installed in other devices comparing a personal computer, a notebook computer, a PDA, a mobile phone, a game player and a video playing and a converting equipment.

30. Three-dimensional virtual input and simulation apparatus of claim 1, wherein the physical input device is one of a mouse, a keyboard, a remote control and a touch screen.