Head rotation tracking from depth-based center of mass

• US 9,098,110 B2
• Filed: 08/18/2011
• Issued: 08/04/2015
• Est. Priority Date: 06/06/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Eichorn, David Mark
• Primary Examiner(s): Park, Edward
• Application Number:
  US13/212,745
• Publication Number:
  US 20120308116A1
• Time in Patent Office: 1,447 Days
• Field of Search: 382/154, 382/100, 382/103
• US Class Current: 1/1
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The rotation of a user'"'"'s head may be determined as a function of depth values from a depth image. In accordance with some embodiments, an area of pixels from a depth image containing a user'"'"'s head is identified as a head region. The depth values for pixels in the head region are used to calculate a center of depth-mass for the user'"'"'s head. The rotation of the user'"'"'s head may be determined based on the center of depth-mass for the user'"'"'s head.

Claims

1. A method for using a depth image to determine a rotation of a user'"'"'s head, the method comprising:

receiving depth image data the depth image data including depth values for a plurality of pixels;

determining a background depth value;

identifying from the depth image data a head region within the depth image;

identifying a subset of pixels in the head region that have a depth value that does not exceed the background depth value;

calculating a center of depth-mass for the user'"'"'s head based on depth values of the subset of pixels in the head region, wherein calculating the center of depth-mass comprises computing a center of mass for a solid of uniform density determined from the depth values for the subset of pixels and the background depth value; and

determining the rotation of the user'"'"'s head based on the location of the center of depth-mass.

2. The method of claim 1, further comprising:

determining the head region by finding an object with a silhouette that closely corresponds with the size and shape of a human body; and

determining a portion of the object that corresponds with the size and shape of a human head.

3. The method of claim 2, wherein the image is captured using a camera other than a red blue green camera.

4. The method of claim 1, wherein determining the rotation of the user'"'"'s head based on the center of depth-mass comprises:

approximating a position in the head region corresponding with a center of the user'"'"'s head; and

determining the rotation of the user'"'"'s head based on a difference between the center of depth-mass and the position in the head region corresponding with the center of the user'"'"'s head.

5. The method of claim 1, wherein the method further comprises using the rotation of the user'"'"'s head to control a camera viewpoint for a virtual environment.

6. A computing device comprising:

a processor configured to;

receive depth values for a plurality of pixels in a depth image;

determine a background depth value;

identify from the depth image data a head region within the depth image;

identify a subset of pixels in the head region that have a depth value that does not exceed the background depth value;

calculate a center of depth-mass for the user'"'"'s head based on depth values of the subset of pixels in the head region, wherein calculating the center of depth-mass comprises computing a center of mass for a solid of uniform density determined from the depth values for the subset of pixels and the background depth value; and

determine the rotation of the user'"'"'s head based on the location of the center of depth-mass.

7. The computing device of claim 6, wherein the device is configured to determine the background depth value by:

identifying a maximum depth value from pixels within the head region; and

setting the background depth value as a function of the maximum depth value.

8. The computing device of claim 6, wherein the device is configured to determine the background depth value by:

calculating an average depth value based on depth values for at least a portion of pixels in the head region; and

setting the background depth value as a function of the average depth value.

9. The computing device of claim 6, wherein the device is configured to identify the head region by:

identifying objects in the depth image based on depth values; and

analyzing the objects to identify a first object having a silhouette that corresponds with the size and shape of a human head.

10. The computing device of claim 6, wherein the device is configured to identify the head region by:

providing a user interface that allows the user to view an image of the user'"'"'s head and to control a location of a boundary for the head region; and

receiving user input setting the location of the boundary for the head region.

11. The computing device of claim 6, wherein the device is configured to identify the head region by:

finding an object with a silhouette that closely corresponds with the size and shape of a human body; and

determining a portion of the object that corresponds with the size and shape of a human head.

12. The computing device of claim 6, wherein the device is configured to determine the rotation of the user'"'"'s head based on the center of depth-mass by:

approximating a position in the head region corresponding with a center of the user'"'"'s head; and

determining the rotation of the user'"'"'s head based on a difference between the center of depth-mass and the position in the head region corresponding with the center of the user'"'"'s head.

13. The computing device of claim 6, wherein the rotation of the user'"'"'s head is used to control a camera viewpoint for a virtual environment.

14. The computing device of claim 6, wherein the device is configured to determine a depth of the user'"'"'s head at each pixel based on the difference between the depth value for each pixel and the background depth value.

15. The computing device of claim 6, wherein the image is captured using a camera other than a red blue green camera.

16. The computing device of claim 6, wherein the image is captured using an infrared camera.

17. One or more computer-storage devices storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method, the method comprising:

receiving depth image data the depth image data including depth values for a plurality of pixels;

determining a background depth value;

identifying from the depth image data a head region within the depth image;

identifying a subset of pixels in the head region that have a depth value that does not exceed the background depth value;

calculating a center of depth-mass for the user'"'"'s head based on depth values of the subset of pixels in the head region, wherein calculating the center of depth-mass comprises computing a center of mass for a solid of uniform density determined from the depth values for the subset of pixels and the background depth value; and

determining the rotation of the user'"'"'s head based on the location of the center of depth-mass.

18. The one or more computer storage devices of claim 17, wherein the center of depth-mass for the user'"'"'s head is calculated using pixels in the head region with a depth value that does not exceed the background depth value.

19. The one or more computer storage devices of claim 17, wherein calculating the center of depth-mass for the user'"'"'s head comprises:

identifying a group of pixels from the head region;

determining a thickness of the user'"'"'s head for each pixel from the group of pixels based on a difference between the depth value for each pixel from the group of pixels and the background depth value;

calculating a first value by summing results from multiplying the thickness of the user'"'"'s head for each pixel from the group of pixels by a position for each pixel from the group of pixels;

calculating a second value by summing the depth values for pixels in the group of pixels; and

calculating the center of depth-mass for the user'"'"'s head by dividing the first value by the second value.

20. The one or more computer storage devices of claim 17, wherein calculating the center of depth-mass for the user'"'"'s head comprises applying weighting as a function of pixel position within the head region.