Systems and methods for detecting a tilt angle from a depth image

• US 8,503,766 B2
• Filed: 12/13/2012
• Issued: 08/06/2013
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mathe, Zsolt, Marais, Charles Claudius
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US13/713,526
• Publication Number:
  US 20130101207A1
• Time in Patent Office: 236 Days
• Field of Search: 382/103, 382/154, 382/190, 382/209, 382/289
• US Class Current: 382/154
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A depth image of a scene may be received, observed, or captured by a device. A human target in the depth image may then be scanned for one or more body parts such as shoulders, hips, knees, or the like. A tilt angle may then be calculated based on the body parts. For example, a first portion of pixels associated with an upper body part such as the shoulders and a second portion of pixels associated with a lower body part such as a midpoint between the hips and knees may be selected. The tilt angle may then be calculated using the first and second portions of pixels.

Claims

1. A method, comprising:

receiving a first and a second frame of depth data, by a computer, the first and the second frames of depth data having been captured from a camera component;

scanning, by the computer, a human target in the first and the second frames of depth data;

calculating, by the computer, a first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data;

calculating, by the computer, a second tilt angle of the human target relative to the camera component based on the scanned human target in the second frame of depth data; and

storing, by the computer, a combined tilt angle for the human target, the combined tilt angle comprising a combination of the first tilt angle and the second tilt angle.

2. The method of claim 1, wherein calculating, by the computer, the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

determining an upper body part and an lower body part of the scanned human target; and

calculating the first tilt angle based on the upper body part and the lower body part of the scanned human target.

3. The method of claim 1, wherein the upper body part comprises the shoulders of the human target, and the lower body part comprises the hips of the human target or the knees of the human target.

4. The method of claim 1, wherein calculating, by the computer, the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

identifying a first portion of pixels of the human target at the location of an upper body part of the human target;

identifying a second portion of pixels of the human target at the location of a lower body part of the human target; and

calculating the first tilt angle of the human target relative to the camera component based on the first and second portions of pixels.

5. The method of claim 4, wherein at least one of the pixels in each of the first portion and second portion of pixels has a depth value associated therewith.

6. The method of claim 4, wherein calculating, by the computer, the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

calculating a first value associated with a vertical distance between the first and second portions; and

calculating the first tilt angle of the human target relative to the camera component based on the first value.

7. The method of claim 4, wherein calculating, by the computer, the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

calculating a first average depth value using the depth value for at least one pixel in the first portion;

calculating a second average depth value using the depth value for at least one pixel in the second portion;

subtracting the second average depth value from the first average depth value to calculate a second value; and

calculating the first tilt angle of the human target relative to the camera component based on the second value.

8. The method of claim 7, wherein calculating the first tilt angle of the human target relative to the camera component based on the second value comprises:

calculating an arctangent of the second value divided by the first value to determine the tilt angle; and

calculating the first tilt angle of the human target relative to the camera component based on the arctangent of the second value.

9. A computer-readable storage device having stored thereon computer executable instructions for detecting a tilt, that, when executed on a computer, cause the computer to perform operations comprising:

receiving a first and a second frame of depth data the first and the second frames of depth data having been captured from a camera component;

scanning a human target in the first and the second frames of depth data;

calculating a first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data;

calculating a second tilt angle of the human target relative to the camera component based on the scanned human target in the second frame of depth data; and

storing a combined tilt angle for the human target, the combined tilt angle comprising a combination of the first tilt angle and the second tilt angle.

10. The computer readable storage device of claim 9, wherein calculating the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

determining an upper body part and an lower body part of the scanned human target; and

calculating the first tilt angle based on the upper body part and the lower body part of the scanned human target.

11. The computer-readable storage device of claim 10, wherein the upper body part comprises shoulders of the human target and the lower body part comprises a midpoint between the hips and the knees of the human target.

12. The computer-readable storage device of claim 9, wherein calculating the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

identifying a first portion of pixels of the human target at the location of an upper body part of the human target;

identifying a second portion of pixels of the human target at the location of a lower body part of the human target; and

calculating the first tilt angle of the human target relative to the camera component based on the first and second portions of pixels.

13. The computer-readable storage device of claim 12, wherein at least one of the pixels in each of the first portion and second portion of pixels has a depth value associated therewith.

14. The computer-readable storage device of claim 12, wherein calculating the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

calculating a first value associated with a vertical distance between the first and second portions; and

calculating the first tilt angle of the human target relative to the camera component based on the first value.

15. The computer-readable storage device of claim 12, calculating the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data comprises:

calculating a first average depth value using the depth value for at least one pixel in the first portion;

calculating a second average depth value using the depth value for at least one pixel in the second portion;

subtracting the second average depth value from the first average depth value to calculate a second value; and

calculating the first tilt angle of the human target relative to the camera component based on the second value.

16. The computer-readable storage device of claim 15, wherein calculating the first tilt angle of the human target relative to the camera component based on the second value comprises:

calculating an arctangent of the second value divided by the first value to determine the tilt angle; and

calculating the first tilt angle of the human target relative to the camera component based on the arctangent of the second value.

17. A system for detecting a tilt, the system comprising:

a memory bearing instructions that, upon execution by a processor, cause the system to at least;

receive a first and a second frame of depth data the first and the second frames of depth data having been captured from a camera component;

scan a human target in the first and the second frames of depth data;

calculate a first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data;

calculate a second tilt angle of the human target relative to the camera component based on the scanned human target in the second frame of depth data; and

store a combined tilt angle for the human target, the combined tilt angle comprising a combination of the first tilt angle and the second tilt angle.

18. The system of claim 17, wherein the instructions that, upon execution by a processor, cause the system to at least calculate the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data further cause the system to at least:

identify a first portion of pixels of the human target at the location of an upper body part of the human target;

identify a second portion of pixels of the human target at the location of a lower body part of the human target; and

calculate the first tilt angle of the human target relative to the camera component based on the first and second portions of pixels.

19. The system of claim 18, wherein the instructions that, upon execution by a processor, cause the system to at least calculate the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data further cause the system to at least:

calculate a first value associated with a vertical distance between the first and second portions; and

calculate the first tilt angle of the human target relative to the camera component based on the first value.

20. The system of claim 18, wherein the instructions that, upon execution by a processor, cause the system to at least calculate the first tilt angle of the human target relative to the camera component based on the scanned human target in the first frame of depth data further cause the system to at least:

calculate a first average depth value using the depth value for at least one pixel in the first portion;

calculate a second average depth value using the depth value for at least one pixel in the second portion;

subtract the second average depth value from the first average depth value to calculate a second value; and

calculate the first tilt angle of the human target relative to the camera component based on the second value.