THREE-DIMENSIONAL POINTING USING ONE CAMERA AND THREE ALIGNED LIGHTS

• US 20150049021A1
• Filed: 12/12/2013
• Published: 02/19/2015
• Est. Priority Date: 08/14/2013
• Status: Active Grant

• Current Assignee: Amchaelvisual Technology LLC
• Original Assignee: Amchael Visual Technology Corporation
• Inventors: Cheng, Kai Michael, Tsai, Yushiuan, Lu, Hsien-Cheng
• Granted Patent: US 9,201,519 B2
• Time in Patent Office: Days
• US Class Current: 345/163
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence

Abstract

Methods and systems for surface-free pointing and/or command input include a computing device operably linked to an imaging device. The imaging device can be any suitable video recording device including a conventional webcam. At least one pointing/input device is provided including first and second sets of visible point light sources, wherein the first and second sets emit differently colored light. The imaging device captures one or more sequential image frames each including a view of a scene including the first and second sets of visible point light sources. One or more software programs calculate a position and/or a motion and/or an orientation of the pointing/input device in the captured image frames by identifying colored areas corresponding to the first set of aligned visible point light sources. Certain activation patterns of individual point light sources are mapped to particular pointing and/or input commands.

Claims

1. A computing system for tracking a motion and/or an orientation of a pointing/input device, comprising:

a computing device having at least one processor, at least one memory, and at least one graphical user interface;

at least one pointing/input device including a first set of aligned visible point light sources emitting light of a wavelength defining a first predetermined color and a second set of visible point light sources emitting light of a wavelength that defines a second predetermined color that is different from the first predetermined color;

an imaging device operably linked to the computing device and configured for capturing a plurality of sequential image frames; and

at least one computer program product operable on the computing device processor, the at least one computer program product including executable instructions for determining at least a position and/or a motion and/or an orientation of the at least one pointing/input device from digital data of the plurality of sequential image frames providing views of the first and second sets of visible point light sources held and/or moved in a three-dimensional space within a field of view of the imaging device, and for rendering on the graphical user interface a visual marker corresponding to the determined position and/or the motion and/or the orientation of the at least one pointing/input device.

2. The system of claim 1, wherein the imaging device is a single lens web cam.

3. The system of claim 1, wherein the at least one computer program product includes executable instructions for mapping and displaying in the graphical user interface a two-dimensional position and/or motion of the at least one pointing/input device within the field of view of the imaging device, comprising:

determining a position of a single actuated point light source of the first set of visible point light sources in each of said plurality of sequential image frames by filtering one or more regions of said plurality of sequential image frames to isolate areas having a predetermined color of the single actuated point light source;

converting the determined position of the single point light source into a position of the visual marker in the graphical user interface; and

displaying the visual marker in the graphical user interface.

4. The system of claim 1, further including mapping an actuation pattern of the first and second sets of visible point light sources to a predetermined pointing and/or input command.

5. The system of claim 1, wherein the at least one computer program product includes executable instructions for mapping and displaying in the graphical user interface a three-dimensional position and/or motion of the at least one pointing/input device, comprising:

determining a three-dimensional coordinate of each visible point light source of the actuated first set of aligned visible point light sources in each of said plurality of sequential image frames;

converting the determined three-dimensional coordinate into a three-dimensional position of the visual marker in the graphical user interface; and

displaying the three-dimensional position of the visual marker in the graphical user interface.

6. The system of claim 5, wherein the at least one computer program product includes executable instructions for determining the three-dimensional coordinate in each of said plurality of sequential image frames by:

calculating a depth of each visible point light source of the actuated first set of aligned visible point light sources with respect to a pre-determined origin of a three-dimensional coordinate system;

calculating an angle of a light beam emitted by each visible point light source of the actuated first set of visible point light sources with respect to a projection plane defining a plane perpendicular to a z-axis of the three-dimensional coordinate system; and

determining a three-dimensional coordinate and orientation of the at least one pointing/input device from said calculated depth and angle.

7. The system of claim 5, further including mapping an actuation pattern of the first and second sets of visible point light sources to a predetermined pointing and/or input command.

8. In a computing system environment, a method for two-dimensional and three-dimensional pointing and/or command input, comprising:

providing at least one pointing/input device including a first set of aligned visible point light sources emitting light of a wavelength defining a first predetermined color and a second set of visible point light sources emitting light of a wavelength defining a second predetermined color that is different from the first predetermined color;

holding and/or moving the at least one pointing/input device in a three-dimensional space disposed within a field of view of an imaging device operably connected to a computing device having at least one processor, at least one memory, and at least one graphical user interface;

by the imaging device, capturing a plurality of sequential image frames each including a view of the at least one pointing/input device, the at least one pointing/input device having at least a portion of the first and second sets of visible point light sources activated to emit light;

from one or more views of at least one of the first set of aligned visible point light sources and the second set of visible point light sources in the captured plurality of sequential image frames, calculating a two-dimensional or a three-dimensional position and/or motion and/or orientation of the at least one pointing/input device; and

rendering a visual marker on the graphical user interface corresponding to the calculated position and/or motion and/or orientation of the at least one pointing/input device.

9. The method of claim 8, further including mapping an actuation pattern of the first and second sets of visible point light sources to a predetermined pointing and/or input command.

10. The method of claim 8, including the steps of:

determining a position of a single actuated point light source of the first set of aligned visible point light sources in each of said plurality of sequential image frames by filtering one or more regions of said plurality of sequential image frames to isolate areas having a predetermined color of the single actuated point light source;

converting the determined position of the single point light source into a position of the visual marker in the graphical user interface; and

displaying the visual marker in the graphical user interface to map and display a two-dimensional position and/or motion of the at least one pointing/input device.

11. The method of claim 8, including mapping and displaying in the graphical user interface a three-dimensional position and/or motion and/or orientation of the at least one pointing/input device, comprising:

determining a three-dimensional coordinate of each visible point light source of the actuated first set of aligned visible point light sources in each of said plurality of sequential image frames;

converting the determined three-dimensional coordinate into a three-dimensional position of the visual marker in the graphical user interface; and

displaying the three-dimensional position of the visual marker in the graphical user interface.

12. The method of claim 11, further including determining the three-dimensional coordinate by:

calculating a depth of each visible point light source of the actuated first set of aligned visible point light sources with respect to a pre-determined origin of a three-dimensional coordinate system;

calculating an angle of a light beam emitted by each visible point light source of the actuated first set of visible point light sources with respect to a projection plane defining a plane perpendicular to a z-axis of the three-dimensional coordinate system; and

determining a three-dimensional coordinate and orientation of the at least one pointing/input device from said calculated depth and angle.

13. A pointing/input device for effecting two-dimensional and three-dimensional pointing and/or command input using a single lens imaging device operatively connected to a computing device having at least one processor, at least one memory, and at least one graphical user interface, comprising:

a housing;

a first set of actuable visible aligned point light sources disposed on a surface of the housing, the first set emitting light of a wavelength defining a first predetermined color;

a second set of actuable visible point light sources disposed on the housing surface, the second set emitting light of a wavelength defining a second predetermined color that is different from the first predetermined color; and

one or more actuators for differently actuating one or more of the visible point light sources of the first and second sets of visible point light sources.

14. The pointing/input device of claim 13, wherein the first predetermined color is blue and the second predetermined color is red.

15. The pointing/input device of claim 13, including an actuator for activating a single visible point light source of the first set.

16. The pointing/input device of claim 15, including an actuator for activating the first set of visible point light sources.

17. The pointing/input device of claim 13, including an actuator for activating a single point light source of the second set.

18. The pointing/input device of claim 17, including an actuator for activating a different single point light source of the second set.