Pose tracking pipeline

• US 8,553,939 B2
• Filed: 02/29/2012
• Issued: 10/08/2013
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Craig, Robert Matthew, Leyvand, Tommer, Peeper, Craig, Al-Ghosien, Momin M., Bronder, Matt, Williams, Oliver, Geiss, Ryan M., Shotton, Jamie Daniel Joseph, Lee, Johnny, Finocchio, Mark
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US13/408,775
• Publication Number:
  US 20120157207A1
• Time in Patent Office: 587 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/123, 382/155, 382/162, 382/168, 382/180, 382/181, 382189-199, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/312, 382/305, 382/154, 375/240.08, 375/240.12
• US Class Current: 382/103
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method of tracking a target includes receiving from a source a depth image of a scene including the human subject. The depth image includes a depth for each of a plurality of pixels. The method further includes identifying pixels of the depth image that belong to the human subject and deriving from the identified pixels of the depth image one or more machine readable data structures representing the human subject as a body model including a plurality of shapes.

Claims

1. A method of computer modeling a human subject, the method comprising:

receiving from a source a depth image of a scene including the human subject, the depth image including a depth for each of a plurality of pixels;

identifying pixels of the depth image that belong to the human subject; and

deriving from the identified pixels of the depth image one or more machine readable data structures representing the human subject as a body model including a plurality of shapes.

2. The method of claim 1, where the plurality of shapes include a plurality of spheres.

3. The method of claim 2, where each of the plurality of spheres is represented by a position variable defining a centroid position of that sphere.

4. The method of claim 3, where each of the plurality of spheres is represented by a size variable defining a size of that sphere.

5. The method of claim 1, where all of the plurality of shapes are a same shape and a same size.

6. The method of claim 1, further comprising deriving from the body model a skeleton including a plurality of skeletal points, each skeletal point having a three-dimensional location.

7. The method of claim 1, where the source includes a structured light depth sensor.

8. The method of claim 1, where the source includes a time-of-flight depth sensor.

9. A method of computer modeling a human subject, the method comprising:

receiving from a source a depth image of a scene including the human subject, the depth image including a depth for each of a plurality of pixels; and

modeling the human subject with a plurality of shapes, each of the plurality of shapes characterized by a machine readable position variable, the machine readable position variable for each shape defining a position of that shape within a volume occupied by the human subject.

10. The method of claim 9, where the plurality of shapes include a plurality of spheres.

11. The method of claim 10, where the machine readable position variable for each of the plurality of spheres defines a centroid position of that sphere.

12. The method of claim 11, where each of the plurality of spheres is represented by a size variable defining a size of that sphere.

13. The method of claim 9, where all of the plurality of shapes are a same shape and a same size.

14. The method of claim 9, further comprising deriving from the plurality of shapes a skeleton including a plurality of skeletal points, each skeletal point having a three-dimensional location.

15. The method of claim 9, where the source includes a structured light depth sensor.

16. The method of claim 9, where the source includes a time-of-flight depth sensor.

17. A data-holding subsystem holding instructions executable by a logic subsystem to:

receive from a source a depth image of a scene including the human subject; and

model the human subject with a plurality of spheres, each of the plurality of spheres characterized by a machine readable position variable, the machine readable position variable for each sphere defining a centroid position of that sphere such that the plurality of spheres collectively occupy a volume of the human subject.

18. The data-holding of claim 17, where all of the plurality of spheres are a same shape and a same size.

19. The data-holding subsystem of claim 17, where the source includes a structured light depth sensor.

20. The data-holding subsystem of claim 17, where the source includes a time-of-flight depth sensor.