Guidance based image photographing device and method thereof for high definition imaging

• US 9,503,632 B2
• Filed: 12/04/2012
• Issued: 11/22/2016
• Est. Priority Date: 12/04/2012
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Yang, Wonkeun, Sohn, Jinho, Lee, Jungho, Yoon, Jeongsuk, Lee, Seongsoo, Hwang, Jeonghwan, An, Shounan, Baek, Seungmin
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US14/649,645
• Publication Number:
  US 20150304549A1
• Time in Patent Office: 1,449 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/175 Static expression
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

An image photographing device for photographing images through a front camera and a rear camera, according to one embodiment of the present invention, comprises: a display unit; a feature extraction unit for extracting facial features from an image of a user'"'"'s face which is displayed on a preview screen through the rear camera; a structure extraction unit for extracting the structure of the user'"'"'s face by using the extracted facial features; an expression extraction unit for extracting the facial expression of the user by using the extracted facial features if the extracted structure of the face matches a standard facial structure; and an notification unit for outputting a photograph notification signal if the extracted facial expression of the user matches a standard facial expression.

Claims

1. An image photographing device photographing an image through a front camera and a rear camera, the image photographing device comprising:

a display;

a feature extractor configured to extract facial features from an image of a user'"'"'s face displayed on a preview screen through the rear camera;

a structure extractor configured to extract a structure of the user'"'"'s face by using the extracted facial features;

an expression extractor configured to extract an expression of the user by using the extracted facial features if the extracted structure of the face matches a standard facial structure; and

an alarm configured to output a photographing notification signal if the extracted expression of the user matches a standard facial expression,

wherein the expression extractor is further configured to;

extract a closed curve of an eye region and a closed curve of a mouth region; and

detect the expression of the user by using a reference point, a length of a long axis and a length of a short axis of the closed curve of the eye region and the closed curve of the mouth region,

wherein the reference point is an intersection point of the long axis and the short axis.

2. The image photographing device according to claim 1, further comprising a structure comparator configured to check whether the extracted structure of the face matches the standard facial structure.

3. The image photographing device according to claim 2, wherein the alarm comprises a guide signal alarm configured to output a guide signal making user'"'"'s motion to enable the structure of the user'"'"'s face to match the standard facial structure, when the extracted structure of the face does not match the standard facial structure.

4. The image photographing device according to claim 1, further comprising an expression comparator configured to check whether the extracted expression of the user matches a standard facial expression.

5. The image photographing device according to claim 4, wherein the alarm comprises a guide signal alarm configured to output a guide signal making user'"'"'s expression to enable the expression of the user to match the standard facial expression, when the extracted expression of the face does not match the standard facial expression.

6. The image photographing device according to claim 3 or 5, wherein the guide signal comprises at least any one of an auditory guide signal and a tactile guide signal.

7. The image photographing device according to claim 1, wherein the feature extractor uses an Eigenface algorithm to extract the facial features.

8. The image photographing device according to claim 7, wherein the feature extractor extracts the contours of the eyes, nose, mouth, ears and hair regions of the user displayed and extracts the facial features.

9. The image photographing device according to claim 2, wherein the structure extractor extracts the structure of the user'"'"'s face through geometric verification and the structure of the user'"'"'s face comprises the angle, position and direction of the user'"'"'s face.

10. The image photographing device according to claim 1, further comprising a memory configured to store the standard facial structure and the standard facial expression, which are based on images photographed through the front camera.

11. An image photographing method of an image photographing device photographing an image through a front camera and a rear camera, the image photographing method comprising:

displaying an image of a user'"'"'s face on the preview screen of a display through the rear camera;

extracting facial features from the image of the user'"'"'s face displayed;

extracting the structure of the user'"'"'s face by using the extracted facial features;

extracting the expression of the user by using the extracted facial features if the extracted structure of the face matches a standard facial structure; and

outputting a photographing notification signal if the extracted expression of the user matches a standard facial expression,

wherein the extracting the expression of the user includes extracting a closed curve of an eye region and a closed curve of a mouth region and

detecting the expression of the user by using a reference point, a length of a long axis and a length of a short axis of the closed curve of the eye region and the closed curve the mouth region,

wherein the reference point is an intersection point of the long axis and the short axis.

12. The image photographing method according to claim 11, further comprising outputting a guide signal making user'"'"'s motion to enable the structure of the user'"'"'s face to match the standard facial structure, when the extracted structure of the face does not match the standard facial structure.

13. The image photographing method according to claim 11, further comprising outputting a guide signal making user'"'"'s expression to enable the expression of the user to match the standard facial expression, when the extracted expression of the face does not match the standard facial expression.

14. The image photographing method according to claim 12 or 13, wherein the outputting of the guide signal comprises outputting at least any one of an auditory guide signal and a tactile guide signal.

15. The image photographing method according to claim 11, wherein the extracting of the facial features comprises using an Eigenface algorithm to extract the facial features through the contours of the eyes, nose, mouth, ears, and hair regions of the user displayed.

16. The image photographing method according to claim 11, wherein the extracting of the structure of the user'"'"'s face comprises extracting the structure of the user'"'"'s face through geometric verification and the structure of the user'"'"'s face comprises the angle, position and direction of the user'"'"'s face.

17. The image photographing device according to claim 11, further comprising automatically photographing an image of a user'"'"'s face displayed on the preview screen of the display when a certain time elapses from when the photographing notification signal is output.

18. The image photographing method according to claim 11, further comprising obtaining the standard facial structure and the standard facial expression through the front camera;

and

storing the obtained standard facial structure and standard facial expression.

19. The image photographing method according to claim 18, wherein the storing the standard facial expression includes storing information on a standard eye expression and information on a standard mouth expression.

20. A non-transitory recording medium having thereon a program for executing the image photographing method according to claim 11.