Apparatus and method for normalizing face image used for detecting drowsy driving

• US 8,055,016 B2
• Filed: 01/19/2006
• Issued: 11/08/2011
• Est. Priority Date: 01/20/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Kwang-Soo, Park, Sang-Cheol, Park, Kyong-Ha
• Primary Examiner(s): Desire; Gregory M
• Assistant Examiner(s): Allison; Andrae S
• Application Number:
  US11/335,216
• Publication Number:
  US 20060188130A1
• Time in Patent Office: 2,119 Days
• Field of Search: 340/575, 340/576, 340/573.1, 340/573.4, 340/691, 340/693, 340/436, 340/439, 340/937, 340/573.7, 382/117, 382/118, 382/203, 382/103, 382/789, 382/104, 351/206, 351/209, 351/246, 351/211, 351/221
• US Class Current: 382/103
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

Disclosed are an apparatus and a method for normalizing an image of a driver'"'"'s face in a predetermined size on the center of a monitor by automatically controlling a lens when the image of the driver'"'"'s face is detected. The apparatus includes a lens for photographing a driver'"'"'s face, a first motor for moving the lens in a forward or rearward direction in order to adjust a zoom parameter, a second motor for moving the lens in a horizontal or vertical direction in order to adjust a pan parameter or a tilt parameter, and a controller for extracting an initial face area from a photographed image and controlling operations of the first motor and the second motor according to the extracted initial face area.

Claims

1. An apparatus for extracting and normalizing a face image to detect a driver'"'"'s state of alertness, the apparatus comprising:

a lens for photographing a driver'"'"'s face;

a first motor for moving the lens in a forward or rearward direction in order to adjust a zoom parameter;

a second motor for moving the lens in a horizontal or vertical direction in order to adjust a pan parameter or a tilt parameter; and

a controller for extracting an initial face area from a photographed driver'"'"'s face image which is an input image and controlling operations of the first motor and the second motor if normalization of the face image is required by analyzing the extracted initial face area, wherein the pan parameter is adjusted by controlling movement of the lens with the second motor until a central x coordinate is positioned between two main vertical lines of the initial face area.

2. The apparatus as claimed in claim 1, further comprising a memory storing frontal face templates used for extracting the initial face image area through matching in the input image.

3. The apparatus as claimed in claim 1, further comprising a memory for storing a look-up table having a zoom parameter adjustment value used to control the first motor and a pan parameter adjustment value and a tilt parameter adjustment value used to control the second motor.

4. The apparatus as claimed in claim 1, wherein at least one of a pan parameter adjustment, a tilt parameter adjustment, and a zoom parameter adjustment is based on a result of face element calculation, the face clement calculation being performed by extracting a horizontal edge from the initial face area extracted from the input image, extracting eye areas and a mouth area, calculating main vertical lines and main horizontal lines, and thereafter calculating a main area.

5. The apparatus as claimed in claim 4, wherein the main vertical lines are determined using coordinate values of at least one right most point in a left eye area of the initial face area and coordinate values of at least one left most point in a right eye area of the initial face area.

6. The apparatus as claimed in claim 4, wherein the main area is determined as a minimum outline square including the eye areas and the mouth area.

7. The apparatus as claimed in claim 6, wherein the main area is defined by means of a first point and a second point, an x coordinate value of the first point is a smaller value selected from between an x coordinate value of a left eye area and an x coordinate value of a left part of the mouth area, a y coordinate value of the first point is a smaller value selected from between a y coordinate value of the left eye area and a y coordinate value of a right eye area, an x coordinate value of the second point is a larger value selected from between an x coordinate value of the right eye area and an x coordinate value of a right part of the mouth area, and a y coordinate value of the second point is a y coordinate value of the mouth area.

8. The apparatus as claimed in claim 1, wherein the tilt parameter adjustment is achieved by controlling movement of the lens with the second motor until a central y coordinate is positioned between two main horizontal lines of the initial face area.

9. The apparatus as claimed in claim 4, wherein the main horizontal lines are determined using coordinate values of lower points in eye areas of the initial face area and coordinate values of upper points in a mouth area of the initial face area.

10. The apparatus as claimed in claim 1, wherein at least one of the pan parameter adjustment, the tilt parameter adjustment, and the zoom parameter adjustment is based on a result of a face element calculation performed by extracting a horizontal edge from the initial face area extracted from the input image, extracting eye areas and a mouth area, and then calculating a main area, the pan parameter adjustment is performed by controlling the second motor such that a center value of x coordinate values in the initial face area is positioned between coordinate values of at least one right point in a left eye area and coordinate values of at least one left point in a right eye area, and the tilt parameter adjustment is performed by controlling the second motor such that a center value of y coordinate values in the initial face area is between coordinate values of at least one lower point in the eye areas and coordinate values of at least one upper point in the mouth area.

11. A method for normalizing face image extraction in an apparatus for extracting the face image used for detecting a driver'"'"'s state of alertness, which includes a camera having lens enabling pan, tilt, and zoom parameter adjustment, the method comprising the steps of:

extracting an initial face area from an input image, via a controller of the apparatus;

calculating face elements based on the extracted initial face area, via the controller of the apparatus;

determining if adjustment of a pan parameter, a tilt parameter, and a zoom parameter are required by analyzing results of the calculated face elements, via the controller of the apparatus; and

obtaining a normalized face image by adjusting at least one of the pan parameter, the tilt parameter, and the zoom parameter according to the determination result, wherein the pan parameter is adjusted by controlling movement of the lens in a predetermined direction until a central x coordinate is positioned between two main vertical lines of the initial face area.

12. The method as claimed in claim 11, wherein, in the step of calculating the face elements includes:

extracting a horizontal edge from an initial face image extracted from the input image;

extracting eye areas and a mouth area from a result of the extracted horizontal edge;

calculating main vertical lines and main horizontal lines based on the extracted eye areas and the extracted mouth area; and

calculating a main area.

13. The method as claimed in claim 12, wherein the main vertical lines are determined using coordinate values of at least one right point in a left eye area of the initial face area and coordinate values of at least one left point in a right eye area of the initial face area.

14. The method as claimed in claim 12, wherein the main horizontal lines are determined using coordinate values of at least one lower point in each of the eye areas of the initial face area and coordinate values of at least one upper point in the mouth area of the initial face area.

15. The method as claimed in claim 11, wherein the tilt parameter adjustment is achieved by controlling the lens in a predetermined direction until a central y coordinate is positioned between two main horizontal lines of the initial face area.

16. The method as claimed in claim 11, wherein at least one of the pan parameter adjustment, the tilt parameter adjustment, and the zoom parameter adjustment is based on a result of a face element calculation performed by extracting a horizontal edge from the initial face area extracted from the input image, extracting eye areas and a mouth area, and then calculating a main area based on the eye areas and the mouth area, the pan parameter adjustment is performed by controlling the second motor such that a center value of x coordinate values in the initial face area is positioned between coordinate values of at least one right point in a left eye area and coordinate values of at least one left point in a right eye area, and the tilt parameter adjustment is performed by controlling the second motor such that a center value of y coordinate values in the initial face area is between coordinate values of at least one lower point in the eye areas and coordinate values of at least one upper point in the mouth area.

17. The method as claimed in claim 11, wherein the main area is determined as a minimum outline square including the eye areas and the mouth area.

18. The method as claimed in claim 17, wherein the main area is defined by a first point and a second point, an x coordinate value of the first point is a smaller value selected from between an x coordinate value ola left eye area and an x coordinate value of a left part of the mouth area, a y coordinate value of the first point is a smaller value selected from between a y coordinate value of the left eye area and a y coordinate value of a right eye area, an x coordinate value of the second point is a larger value selected from between an x coordinate value of the right eye area and an x coordinate value of a right part of the mouth area, and a y coordinate value of the second point has a y coordinate value of the mouth area.