THREE-DIMENSIONAL (3D) HUMAN-COMPUTER INTERACTION SYSTEM USING COMPUTER MOUSE AS A 3D POINTING DEVICE AND AN OPERATION METHOD THEREOF

• US 20140359536A1
• Filed: 10/31/2013
• Published: 12/04/2014
• Est. Priority Date: 06/03/2013
• Status: Active Grant

• Current Assignee: Amchaelvisual Technology LLC
• Original Assignee: Amchael Visual Technology Corporation
• Inventors: Cheng, Kai Michael, Tsai, Yushiuan
• Granted Patent: US 9,310,851 B2
• Time in Patent Office: Days
• US Class Current: 715/848
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/0213 Arrangements providing an integrated pointing device in a keyboard, e.g. trackball, mini-joystick for pointing devices integrated in a laptop computer G06F1/169; joysticks G05G9/047; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/03544 having dual sensing arrangement, e.g. two balls or two coils used to track rotation of the pointing device
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device

Abstract

A three-dimensional user interface system includes at least one pointing/input device and an imaging device configured for capturing one or more image frames each providing at least two different views of a scene including the at least one pointing/input device. The imaging device is a multi-view imaging device which provides at least two different views of the scene per each of the one or more image frames captured. One or more software programs calculate from reference points in the image frames at least a spatial and a velocity parameter of the at least one pointing/input device when moved through a three-dimensional space and for rendering on a graphical user interface of a computing device a visual marker corresponding to the spatial and velocity parameters of the at least one pointing/input device in three-dimensional space. Methods for three-dimensional pointing and/or data input incorporating the described system are also provided.

Claims

1. A three-dimensional user interface system, comprising:

at least one pointing/input device;

an imaging device configured for capturing a plurality of sequential image frames, each of the image frames providing at least two different views of a scene comprising the at least one pointing/input device; and

at least one computer program product operable on a computing device having at least one processor, at least one memory, and at least one graphical user interface;

wherein the at least one computer program product includes executable instructions for calculating from the plurality of sequential image frames at least a spatial and a velocity parameter of the at least one pointing/input device when moved through a three-dimensional space, and for rendering, on the graphical user interface, a visual marker corresponding to the spatial and velocity parameters of the at least one pointing/input device in three-dimensional space.

2. The system of claim 1, wherein the at least one pointing/input device includes at least one visible indicia on a surface thereof providing at least one reference point for the multi-view imaging device.

3. The system of claim 2, wherein the at least one visible indicia is actuable.

4. The system of claim 3, wherein the at least one visible indicia has at least three separate light sources.

5. The system of claim 1, wherein the imaging device is a multi-view imaging device which provides at least two different views of the scene per each of the image frames captured.

6. The system of claim 5, wherein the multi-view imaging device is configured to capture at least three image frames per second.

7. The system of claim 5, wherein the multi-view imaging device is a dual lens video recording device operably linked to an image sensor for converting the captured image frames into digital data.

8. The system of claim 5, wherein the multi-view imaging device is a single lens video recording device operably linked to a reflector for providing the at least two different views of the scene per each of the image frames captured and an image sensor for converting the captured image frames into digital data.

9. The system of claim 5, wherein the computer program product calculates at least a position and a velocity in three-dimensional space of the at least one pointing/input device from the image frames captured by the multi-view imaging device.

10. The system of claim 9, wherein the computer program product includes executable instructions to provide a spatial and velocity parameter of the at least one pointing/input device from the plurality of sequential image frames captured by the multi-view imaging device by at least:

determining a position in the plurality of sequential image frames of the at least one visible indicia when moved through the three-dimensional space;

calculating a normal vector, a bi-normal vector, and a tangent vector of a plane passing through the at least one visible indicia when moved through the three-dimensional space; and

calculating a motion velocity and an acceleration velocity of the at least one visible indicia when moved through the three-dimensional space.

11. In a computing system environment, a method for three-dimensional pointing and/or data input, comprising:

moving at least one pointing/input device in a three-dimensional space within a field of view of an imaging device operably connected to a computing device having at least one processor, at least one memory, and at least one graphical user interface;

by the imaging device, capturing a plurality of sequential image frames with each image frame providing at least two views of a scene including the at least one pointing/input device;

from the captured image frames, calculating at least a spatial and a velocity parameter of the at least one pointing/input device when moved through the three-dimensional space; and

rendering, on the graphical user interface, a visual marker corresponding to the spatial and velocity parameters of the at least one pointing/input device in three-dimensional space.

12. The method of claim 11, including providing at least one computer program product comprising executable instructions operable on at least one computing device having at least one processor and at least one memory, for calculating from the captured image frames at least the spatial and the velocity parameters of the at least one pointing/input device when moved through a three-dimensional space.

13. The method of claim 11, including providing at least one actuable visible indicia on a surface of the at least one pointing/input device.

14. The method of claim 13, wherein the at least one computer program product determines a position in the at least one captured image frame of the at least one visible indicia when moved through the three-dimensional space, calculates a normal vector, a bi-normal vector, and a tangent vector of a plane passing through the at least one visible indicia when moved through the three-dimensional space, and calculates a motion velocity and an acceleration velocity of the at least one visible indicia when moved through the three-dimensional space.

15. The method of claim 11, including providing an imaging device that is a multi-view imaging device configured for capturing the plurality of sequential image frames, each captured image frame providing at least two different views of a scene including the at least one pointing/input device.

16. The method of claim 15, including providing a multi-view imaging device configured to capture at least three image frames per second.

17. The method of claim 14, further including steps of rectifying and calibrating the multi-view imaging device.

18. The method of claim 15, including providing a multi-view imaging device configured to capture at least fifteen image frames per second.

19. The method of claim 15, including providing a multi-view imaging device comprising a dual lens video recording device operably linked to an image sensor for converting the captured image frames into digital data.

20. The method of claim 15, including providing a multi-view imaging device comprising a single lens video recording device operably linked to a reflector for providing the at least two different views of the scene per each of the image frames captured and an image sensor for converting the captured image frames into digital data.