VISUAL TARGET TRACKING

• US 20100197391A1
• Filed: 12/07/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geiss, Ryan M.
• Granted Patent: US 8,682,028 B2
• Time in Patent Office: Days
• US Class Current: 463/30
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A target tracking method includes representing a human target with a machine-readable model including a plurality of skeletal points. The plurality of skeletal points are adjustable into a plurality of different legal poses. The method further includes receiving an observed depth image of the human target from a source and determining proposed positions for one or more of the skeletal points of the machine-readable model based on the observed depth image. The proposed positions of the skeletal points are then adjusted to comply with one or more rules if one or more of the proposed positions violates any of the one or more rules.

Claims

1. A method of target tracking and modeling, the method comprising:

representing a human target with a machine-readable model configured for adjustment into a plurality of different legal poses constrained by one or more rules;

receiving an observed depth image of the human target from a source;

applying forces to force-receiving locations of the machine-readable model based, at least in part, on differences between the observed depth image and the machine-readable model;

determining proposed positions for the force-receiving locations in accordance with the forces applied to the force-receiving locations; and

adjusting the proposed positions of the force-receiving locations to comply with the one or more rules if one or more of the proposed positions violates any of the one or more rules.

2. The method of claim 1, where adjusting the proposed positions of the force-receiving locations includes rectifying the machine-readable model by relaxing the machine-readable model into one of the plurality of different legal poses that is physically expressible by the human target.

3. The method of claim 1, where adjusting the proposed positions of the force-receiving locations includes making iterative adjustments until the machine-readable model complies with the one or more rules.

4. The method of claim 1, where the one or more rules includes one or more of skeletal member length constraints, skeletal point angle constraints, and collision tests.

5. The method of claim 1, where the one or more rules includes skeletal member length constraints, and where adjusting the proposed positions of the force-receiving locations includes applying one or more force vectors to one or more of the force-receiving locations of the machine-readable model to propagate forces along a network of skeletal members of the machine-readable model so as to cause the machine-readable model to settle into one of the plurality of different legal poses.

6. The method of claim 5, where causing the machine-readable model to settle into one of the plurality of different legal poses includes causing the machine-readable model to settle into a legal pose which is least changed from a current illegal pose of the machine-readable model.

7. The method of claim 5, where the skeletal member length constraints include variable length constraints in which one or more skeletal member lengths are variable within a corresponding predetermined range.

8. The method of claim 1, where adjusting the proposed positions of the force-receiving locations to comply with the one or more rules includes adjusting the proposed positions of the force-receiving locations in accordance with relative confidences of the force-receiving locations such that low-confidence force-receiving locations are adjusted relatively more than high-confidence force-receiving locations.

9. The method of claim 1, where the one or more rules includes skeletal point angle constraints, and where adjusting the proposed positions of the force-receiving locations includes restricting a skeletal point angle of a skeletal point of the machine-readable model to lie within a range of acceptable skeletal point angles, the range of acceptable skeletal point angles being based on the plurality of different legal poses that are physically expressible by the human target.

10. The method of claim 1, where the one or more rules includes collision tests, and where adjusting the proposed positions of the force-receiving locations includes applying preventative comparison tests to one or more portions of the machine-readable model to prevent the machine-readable model from interpenetrating itself.

11. A method of target tracking and modeling, the method comprising;

representing a human target with a machine-readable model including a plurality of skeletal points, the plurality of skeletal points adjustable into a plurality of different legal poses;

receiving an observed depth image of the human target from a source;

determining a proposed position for one or more of the skeletal points of the machine-readable model based on the observed depth image;

determining a relative confidence of the proposed position for one or more of the skeletal points of the machine-readable model; and

adjusting the proposed positions of the skeletal points to comply with one or more rules if one or more of the proposed positions violates any of the one or more rules, the proposed positions having relatively low confidences being more readily adjusted than the proposed positions having relatively high confidences.

12. The method of claim 11, where the one or more rules includes one or more of bone length constraints, skeletal point angle constraints and collision tests.

13. The method of claim 11, where the one or more rules includes skeletal member length constraints, and where adjusting the proposed positions of the skeletal points includes adjusting one or more of the proposed positions of the skeletal points such that a length of each of a plurality of bones of the machine-readable model lies within a range of acceptable bone lengths for that bone.

14. The method of claim 11, where the one or more rules includes skeletal point angle constraints, and where adjusting the proposed positions of the skeletal points includes adjusting one or more proposed positions of the skeletal points such that a skeletal point angle of each skeletal point lies within a range of acceptable skeletal point angles.

15. The method of claim 14, where one or more of the ranges of acceptable skeletal point angles is based on the plurality of different legal poses that are physically expressible by the human target.

16. The method of claim 11, where the one or more rules includes collision tests, and where adjusting the proposed positions of the skeletal points includes applying preventative comparison tests to one or more portions of the machine-readable model to prevent the machine-readable model from interpenetrating itself.

17. A method of target tracking and modeling, the method comprising;

representing a human target with a machine-readable model including a plurality of skeletal points, the plurality of skeletal points adjustable into a plurality of different legal poses;

receiving an observed depth image of the human target from a source;

determining proposed positions for one or more of the skeletal points of the machine-readable model based on the observed depth image;

adjusting one or more of the proposed positions of the skeletal points to comply with one or more bone length constraints such that a length of each of a plurality of bones of the machine-readable model lies within a range of acceptable bone lengths for that bone; and

adjusting the proposed positions of the skeletal points to comply with one or more skeletal point angle constraints such that a skeletal point angle of each skeletal point of the machine-readable model lies within a range of acceptable skeletal point angles.

18. The method of claim 17, where one or more of the ranges of acceptable bone lengths is based on the plurality of different legal poses that are physically expressible by the human target.

19. The method of claim 17, where one or more of the ranges of acceptable skeletal point angles is based on the plurality of different legal poses that are physically expressible by the human target.

20. The method of claim 17, further comprising adjusting the proposed positions of the skeletal points by applying preventative comparison tests to one or more portions of the machine-readable model to prevent the machine-readable model from interpenetrating itself.