Upper-body skeleton extraction from depth maps

• US 9,047,507 B2
• Filed: 05/02/2012
• Issued: 06/02/2015
• Est. Priority Date: 05/02/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gurman, Amiad, Tropp, Oren
• Primary Examiner(s): LI, TRACY Y
• Application Number:
  US13/461,802
• Publication Number:
  US 20130293679A1
• Time in Patent Office: 1,126 Days
• Field of Search: 348/46
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A method for processing data includes receiving a depth map of a scene containing at least an upper body of a humanoid form. The depth map is processed so as to identify a head and at least one arm of the humanoid form in the depth map. Based on the identified head and at least one arm, and without reference to a lower body of the humanoid form, an upper-body pose, including at least three-dimensional (3D) coordinates of shoulder joints of the humanoid form, is extracted from the depth map.

Claims

1. A method for processing data, comprising:

receiving a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values;

using a digital processor, processing the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima; and

responsively to the ridge, extracting features of the object from the depth map,

wherein the object is a head of a humanoid form, and wherein extracting the features comprises finding an increase in the respective depth values of the pixels at an upper end of the ridge, and identifying the object as the head responsively to the increase, and

wherein identifying the object as the head comprises processing the depth map so as to locate an arcuate depth edge around the upper end of the ridge.

2. The method according to claim 1, wherein processing the depth map comprises finding each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

3. The method according to claim 1, wherein extracting the features comprises processing the depth map so as to locate one or more depth edges of the object, and applying the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

4. The method according to claim 1, wherein extracting the features comprises partitioning the chain into segments, fitting a line to each segment, and finding a location and orientation of the object based on the line.

5. Apparatus for processing data, comprising:

an imaging assembly, which is configured to capture a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values; and

a processor, which is configured to process the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima, and to extract, responsively to the ridge, features of the object from the depth map,

wherein the object is a head of a humanoid form, and wherein the processor is configured to find an increase in the respective depth values of the pixels at an upper end of the ridge, and to identify the object as the head responsively to the increase, and

wherein the processor is configured to process the depth map so as to locate an arcuate depth edge around the upper end of the ridge, and to identify the object as the head responsively to the arcuate depth edge.

6. The apparatus according to claim 5, wherein the processor is configured to find each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

7. The apparatus according to claim 5, wherein the processor is configured to locate one or more depth edges of the object in the depth map, and to apply the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

8. The apparatus according to claim 5, wherein the processor is configured to partition the chain into segments, to fit a line to each segment, and to find a location and orientation of the object based on the line.

9. A computer software product, comprising a non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values, to process the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima, and to extract, responsively to the ridge, features of the object from the depth map,

wherein the object is a head of a humanoid form, and wherein the instructions cause the computer to find an increase in the respective depth values of the pixels at an upper end of the ridge, and to identify the object as the head responsively to the increase, and

wherein the instructions cause the computer to process the depth map so as to locate an arcuate depth edge around the upper end of the ridge, and to identify the object as the head responsively to the arcuate depth edge.

10. The product according to claim 9, wherein the instructions cause the computer to find each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

11. The product according to claim 9, wherein the instructions cause the computer to locate one or more depth edges of the object in the depth map, and to apply the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

12. The product according to claim 9, wherein the instructions cause the computer to partition the chain into segments, to fit a line to each segment, and to find a location and orientation of the object based on the line.