Fully Automatic Dynamic Articulated Model Calibration
First Claim
1. A computer readable storage device having computer readable software embodied thereon for programming at least one processor to perform a method for modeling a pose of articulated body portions of an object, the method comprising:
- obtaining depth pixels of the articulated body portions in one or more frames;
processing the depth pixels of the one or more frames to identify the articulated body portions of the object;
accessing a model, the model comprising articulated body portions corresponding to the articulated body portions of the object;
matching the articulated body portions of the model to the identified articulated body portions of the object to provide an initial match;
evaluating an extent to which distance constraints are violated in the initial match by the articulated body portions of the model, the distance constraints comprise constraints on distances between the articulated body portions of the model; and
adjusting the model in an attempt to satisfy the distance constraints, including adjusting a length of at least one of the articulated body portions of the model based on the extent to which the distance constraints are violated.
3 Assignments
0 Petitions
Accused Products
Abstract
A depth sensor obtains images of articulated portions of a user'"'"'s body such as the hand. A predefined model of the articulated body portions is provided. The model is matched to corresponding depth pixels which are obtained from the depth sensor, to provide an initial match. The initial match is then refined using distance constraints, collision constraints, angle constraints and a pixel comparison using a rasterized model. Distance constraints include constraints on distances between the articulated portions of the hand. Collision constraints can be enforced when the model meets specified conditions, such as when at least two adjacent finger segments of the model are determined to be in a specified relative position, e.g., parallel. The rasterized model includes depth pixels of the model which are compared to identify overlapping pixels. Dimension of the articulated portions of the model are individually adjusted.
19 Citations
20 Claims
-
1. A computer readable storage device having computer readable software embodied thereon for programming at least one processor to perform a method for modeling a pose of articulated body portions of an object, the method comprising:
-
obtaining depth pixels of the articulated body portions in one or more frames; processing the depth pixels of the one or more frames to identify the articulated body portions of the object; accessing a model, the model comprising articulated body portions corresponding to the articulated body portions of the object; matching the articulated body portions of the model to the identified articulated body portions of the object to provide an initial match; evaluating an extent to which distance constraints are violated in the initial match by the articulated body portions of the model, the distance constraints comprise constraints on distances between the articulated body portions of the model; and adjusting the model in an attempt to satisfy the distance constraints, including adjusting a length of at least one of the articulated body portions of the model based on the extent to which the distance constraints are violated. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A processor-implemented method for modeling a pose of a hand of a user, comprising the processor-implemented steps of:
-
obtaining depth pixels of the hand in one or more frames; processing the depth pixels of the one or more frames to identify articulated portions of the hand; accessing a model of the articulated portions of the hand, the articulated portions of the hand of the model comprising a palm and fingers, including finger segments; matching the articulated portions of the hand of the model to the identified articulated portions of the hand of the depth pixels of the one or more frames, to provide an initial match; evaluating an extent to which distance constraints are violated in the initial match by at least one of the fingers, the distance constraints comprise constraints on distances between finger segments of the at least one of the fingers; rasterizing the model to provide depth pixels of the model; comparing the depth pixels of the at least one of the fingers to the depth pixels of the one or more frames to identify, from among the depth pixels of the one or more frames, non-overlapping depth pixels of the one or more frames which are not overlapping in at least one comparison plane with the depth pixels of the at least one of the fingers of the model; and adjusting the model;
(a) in an attempt to satisfy the distance constraints, including adjusting a length of at least one finger segments of at least one of the fingers of the model based on the extent to which the distance constraints are violated by the at least one of the fingers, and (b) based on the comparing step, to cause the model to more closely match the non-overlapping depth pixels of the one or more frames, by increasing a width of the at least one of the finger segments of the at least one of the fingers of the model. - View Dependent Claims (14, 15, 16)
-
-
17. A processor-implemented method for modeling a pose of a hand of a user, comprising the processor-implemented steps of:
-
obtaining depth pixels of the hand in one or more frames; processing the depth pixels of the one or more frames to identify articulated portions of the hand; accessing a model of the articulated portions of the hand, the articulated portions of the hand of the model comprising a palm and fingers, including finger segments; matching the articulated portions of the hand of the model to the identified articulated portions of the hand of the depth pixels of the one or more frames, to provide an initial match; evaluate an extent to which distance constraints are violated in the initial match by at least one of the fingers, the distance constraints comprise constraints on distances between finger segments of the at least one of the fingers; deciding to enforce collision constraints; in response to the deciding to enforce collision constraints, evaluating an extent to which the collisions constraints are violated in the initial match by at least two adjacent finger segments of the model; and adjusting the model;
(a) in an attempt to satisfy the collision constraints, based on the extent to which the collision constraints are violated by the at least two adjacent finger segments of the model, by decreasing widths of the at least two adjacent finger segments of the model and (b) in an attempt to satisfy the distance constraints, based on the extent to which the distance constraints are violated by the at least one of the fingers. - View Dependent Claims (18, 19, 20)
-
Specification