Pose tracking pipeline

• US 8,860,663 B2
• Filed: 11/22/2013
• Issued: 10/14/2014
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Craig, Robert Matthew, Leyvand, Tommer, Peeper, Craig, Al-Ghosien, Momin M., Bronder, Matt, Williams, Oliver, Geiss, Ryan M., Shotton, Jamie Daniel Joseph, Lee, Johnny, Finocchio, Mark
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/088,184
• Publication Number:
  US 20140078141A1
• Time in Patent Office: 326 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/123, 382/155, 382/162, 382/168, 382180-181, 382189-199, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/312, 382/305, 382/154, 345/158, 345/419, 375/240.08, 375/240.12, 700/47
• US Class Current: 345/158
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method of tracking a subject includes receiving from a source a depth image of a scene including the subject. The depth image includes a depth for each of a plurality of pixels. The method further includes identifying pixels of the depth image that image the subject and deriving from the identified pixels of the depth image one or more machine readable data structures representing the subject as a model including a plurality of shapes.

Claims

1. A method of computer modeling a subject, the method comprising:

receiving from a source a depth image of a scene including the subject, the depth image including a depth for each of a plurality of pixels;

identifying pixels of the depth image that image the subject; and

deriving from the identified pixels of the depth image one or more machine readable data structures representing the subject as a model including a plurality of different geometric shapes.

2. The method of claim 1, where the plurality of different geometric shapes include one or more spheres.

3. The method of claim 2, where each of the one or more spheres is represented by a position variable defining a centroid position of that sphere.

4. The method of claim 3, where each of the one or more spheres is represented by a size variable defining a size of that sphere.

5. The method of claim 1, where each of the plurality of different geometric shapes is represented by one or more size variables defining a size of that shape and one or more position variables defining a position of that shape.

6. The method of claim 1, further comprising deriving from the model a skeleton including a plurality of skeletal points, each skeletal point having a three-dimensional location.

7. The method of claim 1, where the source includes a structured light depth sensor.

8. The method of claim 1, where the source includes a time-of-flight depth sensor.

9. A method of computer modeling a subject, the method comprising:

receiving from a source a depth image of a scene including the subject, the depth image including a depth for each of a plurality of pixels; and

modeling the subject with a plurality of shapes having a same geometric shape and two or more different sizes, each of the plurality of shapes characterized by a machine readable position variable, the machine readable position variable for each shape defining a position of that shape within a volume occupied by the subject.

10. The method of claim 9, where the plurality of shapes are a plurality of spheres.

11. The method of claim 10, where the machine readable position variable for each of the plurality of spheres defines a centroid position of that sphere.

12. The method of claim 10, where each of the plurality of spheres is represented by a size variable defining a size of that sphere.

13. The method of claim 9, further comprising deriving from the plurality of shapes a skeleton including a plurality of skeletal points, each skeletal point having a three-dimensional location.

14. The method of claim 9, where the source includes a structured light depth sensor.

15. The method of claim 9, where the source includes a time-of-flight depth sensor.

16. A data-holding subsystem holding instructions executable by a logic subsystem to:

receive from a source a depth image of a scene including a subject; and

model the subject with a plurality of shapes, each of the plurality of shapes characterized by a machine readable position variable, the machine readable position variable for each shape defining a position of that shape such that the plurality of shapes collectively occupy a volume that approximates the subject.

17. The data-holding subsystem of claim 16, where all of the plurality of shapes are a same geometric shape and a same size.

18. The data-holding subsystem of claim 16, where the source includes a structured light depth sensor.

19. The data-holding subsystem of claim 16, where the source includes a time-of-flight depth sensor.

20. The data-holding subsystem of claim 16, where at least one of the plurality of shapes has a different geometric shape than at least one other of the plurality of shapes.