Method and apparatus for estimating face position in 3 dimensions

• US 8,805,021 B2
• Filed: 08/26/2011
• Issued: 08/12/2014
• Est. Priority Date: 11/17/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hwang, Won Jun, Lee, Ji Hyo, Roh, Kyung Shik, Yoon, Suk June
• Primary Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US13/218,850
• Publication Number:
  US 20120121126A1
• Time in Patent Office: 1,082 Days
• Field of Search: 382/106, 382/115, 382/118, 382/153, 382/154
• US Class Current: 382/106
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

An apparatus and method for estimating a three-dimensional face position. The method of estimating the three-dimensional face position includes acquiring two-dimensional image information from a single camera, detecting a face region of a user from the two-dimensional image information, calculating the size of the detected face region, estimating a distance between the single camera and the user'"'"'s face using the calculated size of the face region, and obtaining positional information of the user'"'"'s face in a three-dimensional coordinate system using the estimated distance between the single camera and the user'"'"'s face. Accordingly, it is possible to estimate the distance between the user and the single camera using the size of the face region of the user in the image information acquired by the single camera so as to acquire the three-dimensional position coordinates of the user.

Claims

1. A method comprising:

acquiring two-dimensional image information from a single camera, the two-dimensional image information including a face of a subject;

detecting a face region of the subject from the two-dimensional image information;

calculating, by a computer, a size of the detected face region;

estimating, by a computer, a distance between the single camera and the face of the subject using the calculated size of the face region; and

obtaining, by a computer, positional information of the face of the subject in a three-dimensional coordinate system using the estimated distance between the single camera and the face of the subject,

wherein the obtaining of the positional information of the face of the subject in the three-dimensional coordinate system includes;

calculating a central position of the detected face region in an image coordinate system; and

converting the central position of the face region in the image coordinate system into a position in the three-dimensional coordinate system using the estimated distance between the single camera and the face of the subject and camera specifications including field of view (FOV) and resolution of the single camera.

2. The method according to claim 1, wherein the converting of the central position of the face region in the image coordinate system into the position in the three-dimensional coordinate system includes:

calculating an angle of projection of the central position of the face region onto a coordinate axis of the image coordinate system based upon the camera specifications; and

converting the central position of the face region in the image coordinate system into the position in the three-dimensional coordinate system using the calculated angle of projection and the estimated distance between the single camera and the face of the subject.

3. The method according to claim 1, wherein the size of the face region is a diagonal distance of the detected face region.

4. The method according to claim 1, wherein the detecting of the face region is performed using Ada-Boost face recognition.

5. The method according to claim 1, wherein the subject is a user of an apparatus.

6. The method according to claim 5, wherein the apparatus is a robot, and the camera is positioned on the robot.

7. The method according to claim 5, wherein the apparatus is an air conditioner system which adjusts services provided by the air conditioner system based on the obtained positional information.

8. A method comprising:

storing reference face sizes according to subject information including age, sex and race information in advance;

acquiring two-dimensional image information from a single camera, the two-dimensional image information including a face of a subject;

detecting a face region of the subject from the two-dimensional image information;

calculating, by a computer, a size of the detected face region;

judging, by a computer, the subject information in the detected face region and selecting a reference face size corresponding to the judged subject information from the reference face sizes;

estimating, by a computer, a distance between the single camera and the face of the subject using the calculated size of the face region and the selected reference face size; and

obtaining, by a computer, positional information of the face of the subject in a three-dimensional coordinate system using the estimated distance between the single camera and the face of the subject,

wherein the obtaining of the positional information of the face of the subject in the three-dimensional coordinate system includes;

calculating a central position of the detected face region in a two-dimensional image coordinate system; and

converting the central position of the face region in the image coordinate system into a position in the three-dimensional coordinate system using the estimated distance between the single camera and the face of the subject and camera specifications including field of view (FOV) and resolution of the single camera.

9. The method according to claim 8, wherein the converting of the central position of the face region in the image coordinate system into the position in the three-dimensional coordinate system includes:

calculating an angle of projection of the central position of the face region onto a coordinate axis of the image coordinate system based upon the camera specifications; and

converting the central position of the face region in the image coordinate system into the position in the three-dimensional coordinate system using the calculated angle of projection and the estimated distance between the single camera and the face of the subject.

10. The method according to claim 8, wherein the size of the face region is a diagonal distance of the detected face region.

11. The method according to claim 8, wherein the detecting of the face region is performed using Ada-Boost face recognition.

12. The method according to claim 8, wherein the subject is a user of an apparatus.

13. The method according to claim 12, wherein the apparatus is a robot, and the camera is positioned on the robot.

14. The method according to claim 12, wherein the apparatus is an air conditioner system which adjusts services provided by the air conditioner system based on the obtained positional information.

15. An apparatus comprising:

a camera including a single camera configured to acquire two-dimensional image information, the two-dimensional image information including a face of a subject;

a face extractor configured to detect a face region of the subject from the two-dimensional image information acquired by the camera;

a face size calculator configured to calculate a size of the face region detected by the face extractor;

a distance estimator configured to estimate a distance between the single camera and the face of the subject using the size of the face region calculated by the face size calculator; and

a three-dimensional position calculator configured to obtain positional information of the face of the subject in a three-dimensional coordinate system using the distance between the camera and the face of the subject estimated by the distance estimator,

wherein the three-dimensional position calculator calculates a central position of the detected face region in an image coordinate system, and converts the calculated central position of the face region into a position in the three-dimensional coordinate system using the estimated distance between the camera and the face of the subject and camera specifications including field of view (FOV) and resolution of the camera.

16. The apparatus according to claim 15, wherein the three-dimensional position calculator calculates an angle of projection of the central position of the face region onto a coordinate axis of the image coordinate system based upon the camera specifications, and converts the central position of the face region into the position in the three-dimensional coordinate system using the calculated angle of projection and the estimated distance between the camera and the face of the subject.

17. The apparatus according to claim 15, wherein the face size calculator calculates a diagonal distance of the face region detected by the face extractor to be used as the size of the face region.

18. The apparatus according to claim 15, wherein the face extractor detects the face region using Ada-Boost face recognition.

19. The apparatus according to claim 17, further comprising:

a memory configured to store reference face sizes according to subject information including age, sex and race information in advance; and

an information judgment calculator configured to judge the subject information using the face region detected by the face extractor,

wherein the distance estimator selects a reference face size corresponding to the subject information judged by the information judgment calculator from the reference face sizes stored in the memory, and estimates the distance between the camera and the face of the subject using the selected reference face size and the calculated size of the face region calculated by the face size calculator.

20. The apparatus according to claim 15, wherein the subject is a user of the apparatus.

21. The apparatus according to claim 20, wherein the apparatus is a robot.

22. The apparatus according to claim 20, wherein the apparatus is an air conditioner system which adjusts services provided by the air conditioner system based on the obtained positional information.