Identification using depth-based head-detection data

• US 9,754,154 B2
• Filed: 12/03/2014
• Issued: 09/05/2017
• Est. Priority Date: 02/15/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Craig, Robert M., Tankovich, Vladimir, Peeper, Craig, Dalal, Ketan, Dedhia, Bhaven, Meekhof, Casey
• Primary Examiner(s): Tucker, Wesley
• Application Number:
  US14/559,757
• Publication Number:
  US 20150086108A1
• Time in Patent Office: 1,007 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06V 40/164 using holistic features

Abstract

A candidate human head is found in depth video using a head detector. A head region of light intensity video is spatially resolved with a three-dimensional location of the candidate human head in the depth video. Facial recognition is performed on the head region of the light intensity video using a face recognizer.

Claims

1. On a computing system, a method comprising:

receiving a depth video;

receiving a light intensity video at least partially spatially-registered to the depth video;

finding a candidate human head in the depth video using a head detector;

spatially resolving a head region of the light intensity video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the light intensity video; and

performing facial recognition on only the head region of the light intensity video using a face recognizer.

2. The method of claim 1, further comprising:

performing skeletal modeling on a body region of the depth video using a body tracker to produce a skeletal model, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

3. The method of claim 2, wherein the skeletal modeling and the facial recognition are performed in parallel.

4. The method of claim 2, further comprising:

responsive to the face recognizer producing a positive identification of a human face in the head region of the light intensity video, associating the skeletal model with the human face.

5. The method of claim 2, wherein the body tracker is configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the head detector.

6. The method of claim 1, wherein the head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

7. The method of claim 1, wherein the face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

8. The method of claim 7, wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

9. A computing system, comprising:

a logic machine; and

a storage machine holding instruction executable by the logic machine to;

receive a depth video;

receive an infrared video at least partially spatially-registered to the depth video;

find a candidate human head in the depth video using a previously-trained, machine-learning head detector;

spatially resolve a head region of the infrared video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the infrared video;

perform facial recognition on only the head region of the infrared video using a previously-trained, machine-learning face recognizer; and

perform skeletal modeling on a body region of the depth video using a previously-trained, machine-learning body tracker to produce a skeletal model, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

10. The computing system of claim 9, wherein the skeletal modeling and the facial recognition are performed in parallel.

11. The computing system of claim 9, wherein the storage machine further holds instructions executable by the logic machine to:

responsive to the previously-trained, machine-learning face recognizer producing a positive identification of a human face in the head region of the infrared video, associate the skeletal model with the human face.

12. The computing system of claim 9, wherein the previously-trained, machine-learning body tracker is configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the previously-trained, machine-learning head detector.

13. The computing system of claim 9, wherein the previously-trained, machine-learning head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

14. The computing system of claim 9, wherein the previously-trained, machine-learning face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan.

15. The computing system of claim 14, wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

16. A computing system, comprising:

a logic machine; and

a storage machine holding instruction executable by the logic machine to;

receive a depth video;

receive an infrared video at least partially spatially-registered to the depth video;

find a candidate human head in the depth video using a previously-trained, machine-learning head detector;

spatially resolve a head region of the infrared video with a three-dimensional location of the candidate human head in the depth video, the head region defining a limited portion of the infrared video;

perform facial recognition on only the head region of the infrared video using a previously-trained, machine-learning face recognizer;

perform skeletal modeling on a body region of the depth video using a previously-trained, machine-learning body tracker to produce a skeletal model, the previously-trained, machine-learning body tracker being configured to constrain a head joint of the skeletal model to the three-dimensional location of the candidate human head as identified by the previously-trained, machine-learning head detector, the body region being spatially contiguous with the three-dimensional location of the candidate human head, and the body region defining a limited portion of the depth video.

17. The computing system of claim 16, wherein the storage machine further holds instructions executable by the logic machine to:

responsive to the previously-trained, machine-learning face recognizer producing a positive identification of a human face in the head region of the infrared video, associate the skeletal model with the human face.

18. The computing system of claim 16, wherein the previously-trained, machine-learning head detector is configured to classify depth pixels of the depth video by producing for each depth pixel a probability that the depth pixel corresponds to a human head without producing a probability that the depth pixel corresponds to another body part, and wherein the candidate human head includes a contiguous region of depth pixels each having a probability this is greater than a threshold.

19. The computing system of claim 16, wherein the previously-trained, machine-learning face recognizer is configured to repeatedly scan the head region inside a bounding rectangle, a size of the bounding rectangle changing each scan, and wherein the bounding rectangle is scaled as a function of a depth of the candidate human head.

20. The computing system of claim 16, wherein finding the candidate human head in the depth video is performed prior to performing facial recognition on the infrared video, and wherein the skeletal modeling and the facial recognition are performed in parallel.