Fully automatic dynamic articulated model calibration
First Claim
1. A computer readable storage device comprising computer readable software embodied thereon for programming at least one processor to perform a method for modeling a pose of articulated body portions of an object, the method comprising:
- obtaining depth pixels of the articulated body portions in one or more frames;
processing the depth pixels of the one or more frames to identify the articulated body portions of the object;
accessing a model, the model comprising articulated body portions corresponding to the articulated body portions of the object;
matching the articulated body portions of the model to the identified articulated body portions of the object to provide an initial match;
evaluating an extent to which distance constraints are violated in the initial match by the articulated body portions of the model, the distance constraints comprise constraints on distances between joints of the articulated body portions of the model; and
adjusting the model in an attempt to satisfy the distance constraints, including adjusting a length of at least one of the articulated body portions of the model based on the extent to which the distance constraints are violated, wherein;
the distance constraints constrain at least two of the articulated body portions of the model to be arranged end to end;
the evaluating comprises evaluating an extent to which the at least two of the articulated body portions of the model are arranged at least one of;
further than end to end in the initial match, or closer than end to end in the initial match; and
the adjusting a length comprises at least one of;
(i) lengthening the at least two of the articulated body portions of the model according to the extent to which the at least two of the articulated body portions of the model are arranged further than end to end in the initial match, or (ii) shortening the at least two of the articulated body portions of the model according to the extent to which the at least two of the articulated body portions of the model are arranged closer than end to end in the initial match.
3 Assignments
0 Petitions
Accused Products
Abstract
A depth sensor obtains images of articulated portions of a user'"'"'s body such as the hand. A predefined model of the articulated body portions is provided. The model is matched to corresponding depth pixels which are obtained from the depth sensor, to provide an initial match. The initial match is then refined using distance constraints, collision constraints, angle constraints and a pixel comparison using a rasterized model. Distance constraints include constraints on distances between the articulated portions of the hand. Collision constraints can be enforced when the model meets specified conditions, such as when at least two adjacent finger segments of the model are determined to be in a specified relative position, e.g., parallel. The rasterized model includes depth pixels of the model which are compared to identify overlapping pixels. Dimension of the articulated portions of the model are individually adjusted.
-
Citations
18 Claims
-
1. A computer readable storage device comprising computer readable software embodied thereon for programming at least one processor to perform a method for modeling a pose of articulated body portions of an object, the method comprising:
-
obtaining depth pixels of the articulated body portions in one or more frames; processing the depth pixels of the one or more frames to identify the articulated body portions of the object; accessing a model, the model comprising articulated body portions corresponding to the articulated body portions of the object; matching the articulated body portions of the model to the identified articulated body portions of the object to provide an initial match; evaluating an extent to which distance constraints are violated in the initial match by the articulated body portions of the model, the distance constraints comprise constraints on distances between joints of the articulated body portions of the model; and adjusting the model in an attempt to satisfy the distance constraints, including adjusting a length of at least one of the articulated body portions of the model based on the extent to which the distance constraints are violated, wherein; the distance constraints constrain at least two of the articulated body portions of the model to be arranged end to end; the evaluating comprises evaluating an extent to which the at least two of the articulated body portions of the model are arranged at least one of;
further than end to end in the initial match, or closer than end to end in the initial match; andthe adjusting a length comprises at least one of;
(i) lengthening the at least two of the articulated body portions of the model according to the extent to which the at least two of the articulated body portions of the model are arranged further than end to end in the initial match, or (ii) shortening the at least two of the articulated body portions of the model according to the extent to which the at least two of the articulated body portions of the model are arranged closer than end to end in the initial match. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A processor-implemented method for modeling a pose of a hand of a user, comprising:
-
obtaining depth pixels of the hand in one or more frames; processing the depth pixels of the one or more frames to identify articulated portions of the hand; accessing a model of the articulated portions of the hand, the articulated portions of the hand of the model comprising a palm and fingers, including finger segments; matching the articulated portions of the hand of the model to the identified articulated portions of the hand of the depth pixels of the one or more frames, to provide an initial match; evaluating an extent to which distance constraints are violated in the initial match by at least one of the fingers, the distance constraints comprise constraints on distances between finger segments of the at least one of the fingers; rasterizing the model to provide depth pixels of the model; comparing the depth pixels of the at least one of the fingers to the depth pixels of the one or more frames to identify, from among the depth pixels of the one or more frames, non-overlapping depth pixels of the one or more frames which are not overlapping in at least one comparison plane with the depth pixels of the at least one of the fingers of the model; and adjusting the model;
(a) in an attempt to satisfy the distance constraints, including adjusting a length of at least one finger segments of at least one of the fingers of the model based on the extent to which the distance constraints are violated by the at least one of the fingers, and (b) based on the comparing, to cause the model to more closely match the non-overlapping depth pixels of the one or more frames, by increasing a width of the at least one of the finger segments of the at least one of the fingers of the model. - View Dependent Claims (12, 13, 14)
-
-
15. A processor-implemented method for modeling a pose of a hand of a user, comprising:
-
obtaining depth pixels of the hand in one or more frames; processing the depth pixels of the one or more frames to identify articulated portions of the hand; accessing a model of the articulated portions of the hand, the articulated portions of the hand of the model comprising a palm and fingers, including finger segments; matching the articulated portions of the hand of the model to the identified articulated portions of the hand of the depth pixels of the one or more frames, to provide an initial match; evaluate an extent to which distance constraints are violated in the initial match by at least one of the fingers, the distance constraints comprise constraints on distances between finger segments of the at least one of the fingers; deciding to enforce collision constraints; in response to the deciding to enforce collision constraints, evaluating an extent to which the collisions constraints are violated in the initial match by at least two adjacent finger segments of the model; and adjusting the model;
(a) in an attempt to satisfy the collision constraints, based on the extent to which the collision constraints are violated by the at least two adjacent finger segments of the model, by decreasing widths of the at least two adjacent finger segments of the model and (b) in an attempt to satisfy the distance constraints, based on the extent to which the distance constraints are violated by the at least one of the fingers. - View Dependent Claims (16, 17, 18)
-
Specification