SYSTEM FOR FINGER RECOGNITION AND TRACKING
First Claim
Patent Images
1. A method for generating a model of a user'"'"'s hand including one or more fingers, comprising:
- (a) receiving position data representing a position of a user interacting with a sensor, the position data including at least one of depth and image data representing the user'"'"'s hand; and
(b) analyzing the position data to identify the user'"'"'s hand in the position data, said step (b) including the steps of;
(b)(1) analyzing depth data from the position data captured in said step (a) to segment the position data into data of the hand, and(b)(2) extracting a set of feature descriptors by applying one or more filters to the image data of the hand identified in said step (b)(1), the one or more filters analyzing image data of the hand as compared to image data outside of a boundary of the hand to discern features of the hand including a shape and orientation of the hand.
2 Assignments
0 Petitions
Accused Products
Abstract
A system and method are disclosed relating to a pipeline for generating a computer model of a target user, including a hand model of the user'"'"'s hands and fingers, captured by an image sensor in a NUI system. The computer model represents a best estimate of the position and orientation of a user'"'"'s hand or hands. The generated hand model may be used by a gaming or other application to determine such things as user gestures and control actions.
-
Citations
20 Claims
-
1. A method for generating a model of a user'"'"'s hand including one or more fingers, comprising:
-
(a) receiving position data representing a position of a user interacting with a sensor, the position data including at least one of depth and image data representing the user'"'"'s hand; and (b) analyzing the position data to identify the user'"'"'s hand in the position data, said step (b) including the steps of; (b)(1) analyzing depth data from the position data captured in said step (a) to segment the position data into data of the hand, and (b)(2) extracting a set of feature descriptors by applying one or more filters to the image data of the hand identified in said step (b)(1), the one or more filters analyzing image data of the hand as compared to image data outside of a boundary of the hand to discern features of the hand including a shape and orientation of the hand. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system for generating a model of a user'"'"'s hand including one or more fingers, the system including a sensing mechanism operatively coupled to a computing device, the system comprising:
-
a skeletal recognition engine for recognizing at least a portion of a skeleton of a user from received data including at least one of image and depth data; an image segmentation engine for segmenting one or more regions of the body into a region representing a hand of the user; and a descriptor extraction engine for extracting data representative of a hand including one or more fingers and an orientation of the hand, the descriptor extraction engine applying a plurality of filters for analyzing pixels in the region representing the hand, each filter in the plurality of filters determining a position and orientation of the hand, the descriptor extraction engine combining the results of each filter to arrive at a best estimate of the position and orientation of the hand. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A computer-readable storage medium not consisting of a modulated data signal, the computer-readable storage medium having computer-executable instructions for programming a processor to perform a method for generating a model of a user'"'"'s hand including one or more fingers for a natural user interface, the method comprising:
-
(a) receiving image data of a user interacting with the natural user interface; (b) analyzing the image data to identify the hand in the image data; and (c) comparing the image data of the identified hand against predefined hand positions to determine if the user has performed one of the following predefined hand gestures or control actions; (c)(1) counting on the user'"'"'s fingers, (c)(2) performing an “
a-okay”
gesture,(c)(3) actuation of a virtual button, (c)(4) pinching together of the thumb and a finger of the hand, (c)(5) writing or drawing, (c)(6) sculpting, (c)(7) puppeteering, (c)(8) turning a knob or combination lock, (c)(9) shooting a gun, (c)(10) performing a flicking gesture, (c)(11) performing a gesture where a finger can be used on an open palm to scroll across and navigate through the virtual space, and (c)(12) moving fingers in a scissor motion to control the legs of a virtual character. - View Dependent Claims (19, 20)
-
Specification