METHOD AND APPARATUS FOR DOING HAND AND FACE GESTURE RECOGNITION USING 3D SENSORS AND HARDWARE NON-LINEAR CLASSIFIERS
First Claim
1. A method for gesture controlling a mobile or stationary terminal comprising a 3D visual and depth sensor using structured light, or multiple stereoscopic image sensors (3D), the method comprising the steps of:
- sensing a hand or face as a portion of the input, isolating these body parts and interpreting the motion gesture or expression being made through a codeless hardware device directly implementing non-linear classifiers to command the terminal to perform a function, similar to a mouse, touch or keyboard entry.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of controlling a mobile or stationary terminal comprising of the steps of one of multiple ways for 3D sensing a hand or face, recognizing the visual command input by trained hardware that does not incorporate instruction based programming and then causing some useful function to be performed by the recognized gesture on the terminal. This method is to enhance gross body gesture recognition in practice today. Gross gesture recognition has been made accessible by providing accurate skeleton tracking information down to the location of a person'"'"'s hands or head. Notably missing from the skeleton tracking data, however, are the detailed positions of the person'"'"'s fingers or facial gestures. Recognizing the arrangement of the fingers on a person'"'"'s hand or expression on his or her face has applications in recognizing gestures such as sign language, as well as user inputs that are normally done with a mouse or a button on a controller. Tracking individual fingers or the subtleties of facial expressions poses many challenges, including the resolution of the depth camera, the possibility for fingers to occlude each other, or be occluded by the hand and performing these functions within the power and performance limitations of traditional coded architectures. This unique codeless, trainable hardware method can recognize finger gestures robustly and deal with these limitations. By recognizing facial expressions, additional information like approval, disapproval, surprise, commands and other useful inputs can be incorporated.
-
Citations
9 Claims
-
1. A method for gesture controlling a mobile or stationary terminal comprising a 3D visual and depth sensor using structured light, or multiple stereoscopic image sensors (3D), the method comprising the steps of:
- sensing a hand or face as a portion of the input, isolating these body parts and interpreting the motion gesture or expression being made through a codeless hardware device directly implementing non-linear classifiers to command the terminal to perform a function, similar to a mouse, touch or keyboard entry.
- View Dependent Claims (2, 3, 4, 5, 6, 7)
-
8. A system where there is no CPU or encoded instruction processing unit directly connected with the sensors and the output of the RGB sensor and IR depth sensor are directed into the hardware based non-linear classifier. This configuration which may also include external memory and an FPGA, wherein the hardware based nonlinear classifier takes the image information and directly recognizes the hand or face gesture and commands the terminal CPU to perform a function.
-
9. A system where there is no CPU or encoded instruction processing unit with the sensors and the output of the two CMOS image sensors (stereoscopic for depth) are directed into the hardware based non-linear classifier which may also include external memory and an FPGA, wherein the hardware based nonlinear classifier takes the image information and directly recognizes the hand or face gesture and commands the terminal cpu to perform a function.
Specification