THREE-DIMENSIONAL HUMAN-COMPUTER INTERACTION SYSTEM THAT SUPPORTS MOUSE OPERATIONS THROUGH THE MOTION OF A FINGER AND AN OPERATION METHOD THEREOF

• US 20130069876A1
• Filed: 09/19/2011
• Published: 03/21/2013
• Est. Priority Date: 09/19/2011
• Status: Active Grant

• Current Assignee: Amchaelvisual Technology LLC
• Original Assignee: Amchael Visual Technology Corporation
• Inventors: Cheng, Fuhua, Chang, Lu Hsien
• Granted Patent: US 8,648,808 B2
• Time in Patent Office: Days
• US Class Current: 345/166
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A three-dimensional (3D) human-computer interaction system that supports mouse operations through the motion of a finger and an operation method thereof are provided. In the provided system and method, a multi-view video recording device captures an image of a finger of an operator, and has an information connection with an electronic information device through an interaction handling program. After the interaction handling program is executed, a CPU of the electronic information device performs operations such as computation, synthesis, image presentation, gesture tracking, and command recognition on the captured image to interpret a motion of the finger of the user. Accordingly, the user may perform operations on an operation interface of the electronic information device by using the finger in a 3D space in a noncontact mode. The present invention is particularly applicable to application software with 3D space operations as it can perform 3D interaction with the application software.

Claims

1. A three-dimensional (3D) human-computer interaction system that supports mouse operations through the motion of a finger, capable of being installed in an electronic information device connected with a display device, so that an operator is enabled to perform a noncontact operation at an interaction interface of the electronic information device with a motion of a finger thereof, the system comprising:

a multi-view video recording device, having an information connection with the electronic information device, and comprising an image sensor module and a multi-view image capturing module, wherein the multi-view image capturing module forms a left image capturing path, a central image capturing path, and a right image capturing path, so that the image sensor module synchronously captures multi-view images of the motion of the finger of the operator respectively with the multi-view image capturing module; and

an interaction handling program, installed in an operating system of the electronic information device, and used for driving the multi-view video recording device to operate after an initialization operation, the interaction handling program comprising a computation unit for computing the image acquired by the multi-view video recording device and generating image information, a 3D imaging unit for establishing a model of a 3D image for the image information, a gesture tracking unit for tracking the motion of the finger of the operator, and a command recognition unit for recognizing the acquired motion of the finger to map the motion to a preset operation command.

2. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 1, wherein the left image capturing path of the multi-view image capturing module is formed of a left image capturing opening, a left first reflecting surface, and a left second reflecting surface.

3. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 1, wherein the right image capturing path of the multi-view image capturing module is formed of a right image capturing opening, a right first reflecting surface, and a right second reflecting surface.

4. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 1, wherein the interaction handling program has an image presentation unit capable of presenting the 3D image established by the 3D imaging unit at the display device.

5. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 4, wherein the display device is a 3D display device.

6. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 1, wherein the image sensor module is a complementary metal-oxide-semiconductor (CMOS) sensor module.

7. The 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 1, wherein the image sensor module is a charge-coupled device (CCD) sensor module.

8. An operation method of a three-dimensional (3D) human-computer interaction system that supports mouse operations through the motion of a finger, wherein a multi-view video recording device performs an image capture operation in a synchronous and multi-view manner, and an interaction handling program executes computation and an operation command, the operation method comprising:

starting an operation mode, wherein the interaction handling program starts an operation mode, so that an electronic information device enters the operation mode;

recognizing a body feature, wherein the multi-view video recording device synchronously captures an image of a body feature applied by an operator in multi-view directions, and each captured image is recorded and recognized by the interaction handling program;

acquiring a first position, wherein after the above steps, the interaction handling program acquires a first position of the body feature after the recognition;

tracking the body feature, wherein by using the body feature of the acquired first position as a basic parameter, the multi-view video recording device continuously captures images of the same body feature to achieve an objective of tracking, and when the body feature starts displacement, a next position occurs right away;

computing displacement and a depth, wherein when the body feature moves from the previous position to the next position, the interaction handling program synchronously computes displacement and a depth for the process of position changes of the body feature;

recognizing the body feature and mapping a command, wherein after the computation of the displacement and the depth, the acquired information of the position changes of the body feature is mapped to a pre-defined operation command; and

executing the command, wherein the electronic information device operates according to the mapped operation command.

9. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the step of starting the operation mode comprises starting a two-dimensional (2D) operation mode.

10. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the step of starting the operation mode comprises starting a 3D operation mode.

11. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein in the step of tracking the body feature, the body feature is any one of a finger, a number of fingers, a finger-like body, a skin color, a shape, a joint position, or a joint motion change.

12. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein in the step of tracking the body feature, after the next position occurs, the previous position is recorded and used as a relative reference for the next position.

13. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein in the step of computing the displacement and the depth, after the depth computation, a depth map of image depth information is generated.

14. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the pre-defined operation command is any one of moving left, moving right, moving up, moving down, moving forward, and moving backward.

15. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the pre-defined operation command is either a single-click or a double-click.

16. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the pre-defined operation command is dragging.

17. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the pre-defined operation command is a 3D move command.

18. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the pre-defined operation command is a 3D rotation command.

19. The operation method of the 3D human-computer interaction system that supports mouse operations through the motion of the finger according to claim 8, wherein the step of starting the operation mode comprises starting a “

touch-like”

operation mode.