Image object detection using separate ranges from both image detections

• US 8,457,351 B2
• Filed: 08/23/2007
• Issued: 06/04/2013
• Est. Priority Date: 02/23/2005
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Ito, Hisao, Baba, Kozo, Tafuku, Akiyoshi, Saito, Taku, Tohno, Masatoshi, Katagiri, Taku
• Primary Examiner(s): ZARKA, DAVID PETER
• Application Number:
  US11/844,097
• Publication Number:
  US 20070291999A1
• Time in Patent Office: 2,112 Days
• Field of Search: 382/118, 382/103
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

Provided are an image processing method, an image processing system, an image processing device and a computer program for detecting a detection object such as nares of the driver with a high degree of accuracy in, for example, a system using an in-vehicle camera which is mounted on a vehicle and takes an image of the face of the driver. A detection object is diversified by a variety of detection methods such as a method for detecting a plurality of locations in the vertical direction as candidates during image pickup of an image, detecting a range to be a candidate of a detection object on the basis of the brightness of a pixel for each of rows of pixels lined up in the horizontal direction corresponding to each detected location and specifying a detection object from candidates of a detection object on the basis of the length of the detected range.

Claims

1. An image processing method of detecting a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, comprising:

obtaining the two-dimensional image;

integrating brightness of pixels of the two-dimensional image lined up in the first direction;

deriving a change in an integrated value of the two-dimensional image in the second direction;

obtaining a quadratic differential value of the derived change;

detecting a plurality of locations in the second direction, which indicate minimum values in ascending order of the obtained quadratic differential value among the integrated brightness of pixels in the first direction, as locations corresponding to candidates of a detection object on the basis of the derived change in an integrated value;

detecting a range in the first direction based on a change of brightness of a pixel lined up in the first direction as a candidate of the detection object for each of rows of pixels lined up in the first direction corresponding to each detected location; and

specifying the detection object from candidates of the detection object on the basis of length of the detected range.

2. An image processing method of detecting a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, comprising:

obtaining the two-dimensional image;

integrating numerical values in the second direction, said numerical values being based on a change in brightness of pixels of the two-dimensional image lined up in the first direction;

deriving a change in an integrated value of the two-dimensional image in the first direction;

integrating numerical values in the first direction, said numerical values being based on a change in brightness of pixels lined up in the second direction;

deriving a change in an integrated value in the second direction;

obtaining a quadratic differential value of the derived changed in the second direction; and

detecting a detection object on the basis of both a range in the first direction on the basis of the derived change in an integrated value in the first direction and a range in the second direction on the basis of the obtained quadratic differential value.

3. An image processing device which detects a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, comprising:

an obtaining part which obtains the two-dimensional image;

a deriving part which integrates brightness of pixels of the two-dimensional image lined up in the first direction and derives a change in an integrated value of the two-dimensional image in the second direction;

an obtaining part which obtains a quadratic differential value of the derived change;

a candidate detecting part which detects a plurality of locations in the second direction, which indicate minimum values in ascending order of the obtained quadratic differential value among the integrated brightness of pixels in the first direction, as locations corresponding to candidates of a detection object on the basis of the derived change in the integrated value;

a range detecting part which detects a range in the first direction based on a change of brightness of a pixel lined up in the first direction as a candidate of the detection object for each of rows of pixels lined up in the first direction corresponding to each of the plurality of detected locations; and

a specifying part which specifies the detection object from candidates of the specific detection object on the basis of length of the detected range.

4. The image processing device according to claim 3, wherein the candidate detecting part detects the plurality of locations for which a change in an integrated value of brightness of pixels integrated in the first direction indicates a local minimum value.

5. The image processing device according to claim 4,

further comprising a part which obtains a quadratic differential value of the change in an integrated value derived by the deriving part,

wherein the candidate detecting part detects a predetermined number of locations in the plurality of locations indicating a local minimum value, on the basis of a result of quadratic differential.

6. The image processing device according to claim 3, wherein the range detecting part detects a range on the basis of a change in brightness of pixels lined up in the first direction.

7. The image processing device according to claim 3,

further comprising a part which detects a range in the first direction of a detection area which includes the detection object and has a range in the first direction larger than the detection object,

wherein the specifying part specifies the detection object on the basis of a result of comparison between length of a range in the first direction of a candidate of the detection object detected by the range detecting part and length of a range in the first direction of a detection area including the detection object.

8. The image processing device according to claim 3, further comprising:

a part which integrates brightness of pixels lined up in the second direction according to the specified detection object and derives a change in an integrated value in the first direction;

a local minimum value detecting part which detects a local minimum value from a change in an integrated value in the first direction;

a part which counts the number of detected local minimum values; and

a part which determines that the specified detection object is false when the counted number is smaller than a predetermined number.

9. An image processing system, comprising:

an image processing device according to claim 3; and

an image pickup device which generates an image to be processed by the image processing device,

wherein the detection object is an area including nares of a person in an image taken by the image pickup device, the first direction is a horizontal direction, and the second direction is a vertical direction.

10. An image processing device which detects a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, comprising:

obtaining the two-dimensional image;

a first deriving part which integrates numerical values, in the second direction, based on a change in brightness of pixels of the two-dimensional image lined up in the first direction and derives a change in an integrated value of the two-dimensional image in the first direction;

a second deriving part which integrates numerical values, in the first direction, based on a change in brightness of pixels lined up in the second direction and derives a change in an integrated value in the second direction an obtaining part which obtains a quadratic differential value of the derived change in the second direction; and

a detecting part which detects a detection object on the basis of both a range in the first direction based on the change in an integrated value in the first direction derived by the first deriving part and a range in the second direction based on the obtained quadratic differential value.

11. The image processing device according to claim 10,

wherein the first deriving part integrates indexes, in the second direction, based on both a numerical value based on a brightness difference from an adjoining pixel in the first direction and a numerical value indicating a low level of brightness of the adjoining pixel in the first direction and derive a change in an integrated value in the first direction,

wherein the second deriving part integrates indexes, in the first direction, based on both a numerical value based on a brightness difference from an adjoining pixel in the second direction and a numerical value indicating a low level of brightness of the adjoining pixel in the second direction and derive a change in an integrated value in the second direction, and

wherein the detecting part detects a detection object on the basis of both a range in the first direction from a location where the integrated value derived by the first deriving part becomes maximum to a location where the integrated value derived by the first deriving part becomes minimum and a range in the second direction from a location where the integrated value derived by the second deriving part becomes maximum to a location where the integrated value derived by the second deriving part becomes minimum.

12. An image processing system, comprising:

an image processing device according to claim 10; and

an image pickup device which generates an image to be processed by the image processing device,

wherein the detection object is an area including nares of a person in an image taken by the image pickup device, the first direction is a horizontal direction, and the second direction is a vertical direction.

13. A non-transitory computer-readable memory product storing a computer program that when executed causes a computer to perform a method of detecting a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, said method comprising:

obtaining by the computer the two-dimensional image;

integrating in the computer brightness of pixels of the two-dimensional image lined up in the first direction;

deriving by the computer a change in an integrated value of the two-dimensional image in the second direction;

obtaining by the computer a quadratic differential value of the derived change;

detecting by the computer a plurality of locations in the second direction, which indicate minimum values in ascending order of the obtained quadratic differential value among the integrated brightness of pixels in the first direction, as locations corresponding to candidates of a detection object on the basis of the derived change in an integrated value;

detecting by the computer a range in the first direction based on a change of brightness of a pixel lined up in the first direction as a candidate of the detection object for each of rows of pixels lined up in the first direction corresponding to each detected location; and

specifying by the computer the detection object from candidates of the detection object on the basis of length of the detected range.

14. A non-transitory computer-readable memory product storing a computer program that when executed causes a computer to perform a method of detecting a detection object from a two-dimensional image in which pixels are lined up in both a first direction and a second direction that are different from each other, said method comprising:

obtaining by the computer the two-dimensional image;

integrating in the computer numerical values, in the second direction, based on a change in brightness of pixels of the two-dimensional image lined up in the first direction;

deriving by the computer a change in an integrated value of the two-dimensional image in the first direction;

integrating in the computer numerical values, in the first direction, based on a change in brightness of pixels lined up in the second direction;

deriving by the computer a change in an integrated value in the second direction;

obtaining by the computer a quadratic differential value of the derived change in the second direction; and

detecting by the computer a detection object on the basis of both a range in the first direction based on the derived change in an integrated value in the first direction and a range in the second direction based on the obtained quadratic differential value.