Identification using depth-based head-detection data

• US 10,552,666 B2
• Filed: 08/01/2017
• Issued: 02/04/2020
• Est. Priority Date: 02/15/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Craig, Robert M., Tankovich, Vladimir, Peeper, Craig, Dalal, Ketan, Dedhia, Bhaven, Meekhof, Casey
• Primary Examiner(s): Tucker, Wesley J
• Application Number:
  US15/666,113
• Publication Number:
  US 20180075288A1
• Time in Patent Office: 917 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06V 40/164 using holistic features

Abstract

A candidate human head is found in depth video using a head detector. A head region of light intensity video is spatially resolved with a three-dimensional location of the candidate human head in the depth video. Facial recognition is performed on the head region of the light intensity video using a face recognizer.

Claims

1. On a computing system, a method comprising:

receiving a depth video;

receiving a light intensity video at least partially spatially-registered to the depth video;

finding a candidate human head in the depth video;

spatially resolving a head region of the light intensity video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the light intensity video; and

performing facial recognition focused on the head region of the light intensity video.

2. The method of claim 1, further comprising:

performing skeletal modeling on a body region of the depth video, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

3. The method of claim 2, wherein the skeletal modeling and the facial recognition are performed in parallel.

4. The method of claim 2, further comprising:

responsive to a positive identification of a human face in the head region of the light intensity video, associating the skeletal model with the human face.

5. The method of claim 2, further comprising constraining a head joint of the skeletal model to the three-dimensional location of the candidate human head.

6. The method of claim 1, further comprising classifying depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

7. The method of claim 1, further comprising repeatedly scanning the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

8. The method of claim 7, wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

9. The method of claim 1, wherein the light-intensity video is an infrared video.

10. A computing system, comprising:

a logic machine; and

a storage machine holding instruction executable by the logic machine to;

receive a depth video;

receive a light intensity video at least partially spatially-registered to the depth video;

find a candidate human head in the depth video using a head detector;

spatially resolve a head region of the light intensity video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the light intensity video; and

perform facial recognition focused on the head region of the light intensity video.

11. The computing system of claim 10, wherein the instructions are executable by the logic machine to:

perform skeletal modeling on a body region of the depth video, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

12. The computing system of claim 11, wherein the skeletal modeling and the facial recognition are performed in parallel.

13. The computing system of claim 11, wherein the instructions are executable by the logic machine to associate the skeletal model with a human face responsive to a positive identification of the human face in the head region of the light intensity video.

14. The computing system of claim 11, wherein the instructions are executable by the logic machine to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head.

15. The computing system of claim 10, wherein the instructions are executable by the logic machine to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

16. The computing system of claim 10, wherein the instructions are executable by the logic machine to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

17. The computing system of claim 16, wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

18. The computing system of claim 10, wherein the light-intensity video is an infrared video.

19. On a computing system, a method comprising:

receiving a depth image;

receiving a light intensity image at least partially spatially-registered to the depth image;

finding a candidate human head in the depth image; and

performing facial recognition focused on a candidate portion of the light intensity image spatially resolved to where the candidate human head is found in the depth image.

20. The method of claim 19, wherein the candidate portion of the light intensity image is defined by a bounding rectangle scaled as a function of a depth of the candidate human head.