Three dimensional (3D) modeling of a complex control object
First Claim
1. A method of initializing predictive information that models a complex control object in a three dimensional (3D) sensory space, the method including:
- accessing observed information including a set of contour points corresponding to surface points at along an outline of a complex control object in a three dimensional (3D) sensory space;
transforming the set of contour points to a normalized orientation of the control object, including;
at training time t0, sensing an actual position of at least one complex control object in a first reference frame of the 3D sensory space;
at initialization time t1, sensing, in the 3D sensory space, an apparent position of the complex control object different from the actual position, wherein the complex control object has not moved in the 3D sensory space between t0 and t1;
calculating a second reference frame that accounts for apparent position of the complex control object;
calculating a transformation that renders the actual position in the first reference frame and the apparent position in the second reference frame into a common reference frame; and
transforming the actual and apparent positions of the complex control object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed apparent position is transformed to an actual position;
searching a plurality of observed information archetypes that represent poses of the control object in the normalized orientation, the poses including arrangement of features of the complex control object and a perspective of observing the complex control object, and selecting an archetype; and
initializing predictive information that models the complex control object from initialization parameters associated with the selected archetype.
12 Assignments
0 Petitions
Accused Products
Abstract
The disclosed technology automatically (e.g., programmatically) initializes predictive information for tracking a complex control object (e.g., hand, hand and tool combination, robot end effector) based upon information about characteristics of the object determined from sets of collected observed information. Automated initialization techniques obviate the need for special and often bizarre start-up rituals (place your hands on the screen at the places indicated during a full moon, and so forth) required by conventional techniques. In implementations, systems can refine initial predictive information to reflect an observed condition based on comparison of the observed with an analysis of sets of collected observed information.
-
Citations
28 Claims
-
1. A method of initializing predictive information that models a complex control object in a three dimensional (3D) sensory space, the method including:
-
accessing observed information including a set of contour points corresponding to surface points at along an outline of a complex control object in a three dimensional (3D) sensory space; transforming the set of contour points to a normalized orientation of the control object, including; at training time t0, sensing an actual position of at least one complex control object in a first reference frame of the 3D sensory space; at initialization time t1, sensing, in the 3D sensory space, an apparent position of the complex control object different from the actual position, wherein the complex control object has not moved in the 3D sensory space between t0 and t1; calculating a second reference frame that accounts for apparent position of the complex control object; calculating a transformation that renders the actual position in the first reference frame and the apparent position in the second reference frame into a common reference frame; and transforming the actual and apparent positions of the complex control object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed apparent position is transformed to an actual position; searching a plurality of observed information archetypes that represent poses of the control object in the normalized orientation, the poses including arrangement of features of the complex control object and a perspective of observing the complex control object, and selecting an archetype; and initializing predictive information that models the complex control object from initialization parameters associated with the selected archetype. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)
-
-
27. A system of initializing predictive information that models a complex control object in a three dimensional (3D) sensory space, the system including:
-
a camera; and one or more processors coupled to the camera and to a non-transitory computer readable medium storing instructions thereon, which instructions when executed by the processors perform; accessing observed information including a set of contour points corresponding to surface points at along an outline of a complex control object in a three dimensional (3D) sensory space; transforming the set of contour points to a normalized orientation of the control object, including; at training time t0, sensing an actual position of at least one complex control object in a first reference frame of the 3D sensory space; at initialization time t1, sensing, in the 3D sensory space, an apparent position of the complex control object different from the actual position, wherein the complex control object has not moved in the 3D sensory space between t0 and t1; calculating a second reference frame that accounts for apparent position of the complex control object; calculating a transformation that renders the actual position in the first reference frame and the apparent position in the second reference frame into a common reference frame; and transforming the actual and apparent positions of the complex control object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed apparent position is transformed to an actual position; searching a plurality of observed information archetypes that represent poses of the control object in the normalized orientation, the poses including arrangement of features of the complex control object and a perspective of observing the complex control object, and selecting an archetype; and initializing predictive information that models the complex control object from initialization parameters associated with the selected archetype.
-
-
28. A non-transitory computer readable memory containing computer program instructions for initializing predictive information that models a complex control object in a three dimensional (3D) sensory space, which instructions when executed by one or more processors perform:
-
accessing observed information including a set of contour points corresponding to surface points at along an outline of a complex control object in a three dimensional (3D) sensory space; transforming the set of contour points to a normalized orientation of the control object, including; at training time t0, sensing an actual position of at least one complex control object in a first reference frame of the 3D sensory space; at initialization time t1, sensing, in the 3D sensory space, an apparent position of the complex control object different from the actual position, wherein the complex control object has not moved in the 3D sensory space between t0 and t1; calculating a second reference frame that accounts for apparent position of the complex control object; calculating a transformation that renders the actual position in the first reference frame and the apparent position in the second reference frame into a common reference frame; and transforming the actual and apparent positions of the complex control object into the common reference frame, wherein the common reference frame has a fixed point of reference and an initial orientation of axes, whereby the sensed apparent position is transformed to an actual position; searching a plurality of observed information archetypes that represent poses of the control object in the normalized orientation, the poses including arrangement of features of the complex control object and a perspective of observing the complex control object, and selecting an archetype; and initializing predictive information that models the complex control object from initialization parameters associated with the selected archetype.
-
Specification