DETECTING OBJECTS IN AN IMAGE BEING ACQUIRED BY A DIGITAL CAMERA OR OTHER ELECTRONIC IMAGE ACQUISITION DEVICE

• US 20110205387A1
• Filed: 05/02/2011
• Published: 08/25/2011
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Zoran Corporation (Qualcomm, Inc.)
• Inventors: Shaick, Ben-Zion, Pinto, Victor, Dvir, Itsik, Tzur, Meir
• Granted Patent: US 8,379,922 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

The likelihood of a particular type of object, such as a human face, being present within a digital image, and its location in that image, are determined by comparing the image data within defined windows across the image in sequence with two or more sets of data representing features of the particular type of object. The evaluation of each set of features after the first is preferably performed only on data of those windows that pass the evaluation with respect to the first set of features, thereby quickly narrowing potential target windows that contain at least some portion of the object. Correlation scores are preferably calculated by the use of non-linear interpolation techniques in order to obtain a more refined score. Evaluation of the individual windows also preferably includes maintaining separate feature set data for various positions of the object around one axis and rotating the feature set data with respect to the image data for the individual windows about another axis.

Claims

1. A method comprising:

defining a plurality of windows within an image;

assigning scores to individual windows of the plurality of windows by

evaluating data of the image within the individual windows with respect to stored data of a set of features of a particular type of object,

for each individual window, determining relationships between a result of said evaluating and a plurality of different thresholds, and

assigning a score to each individual window based on the determined relationships for said window; and

determining a likelihood that the particular type of object is present in the image based on the assigned scores.

2. The method of claim 1, wherein said assigning scores comprises:

for each individual window, comparing the result of said evaluating against a first threshold and a second threshold;

if the result is above a higher of the first and second thresholds, assigning a first value to the score;

if the result is below a lower of the first and second thresholds, assigning a second value to the score; and

if the result is between the first and second thresholds, assigning to the score a value interpolated between the first and second values.

3. The method of claim 2, wherein assigning to the score a value interpolated between the first and second values comprises performing interpolation between the first and second values non-linearly by a transfer function formed of at least two parabolic functions.

4. The method of claim 1, wherein assigning a score to a current window, of the plurality of windows, comprises comparing a dot product of a vector of the image projected onto the current window with a vector of the current window.

5. The method of claim 4, wherein:

the dot product is represented by F(ν

_i, I), where ν

_i is a vector projected onto the current window I,

the first and second thresholds are represented by θ

₀ and θ

₁, respectively, where θ

₁>

θ

₀,

the first and second values are predetermined values represented by α and

β

, respectively, where β

>

α

, and

the score is interpolated between the first and second predetermined values when θ

₀<

F(ν

_i, I)<

θ

₁ and is represented by (α

+(F(ν

_i, I)−

θ

₀)(β

−

α

)/(θ

₁−

θ

₀)).

6. The method of claim 4, wherein:

the dot product is represented by F(ν

_i, I), where ν

_i is a vector projected onto the current window I,

the first and second thresholds are represented by θ

₀ and θ

₁, respectively, where θ

₁>

θ

₀,

the first and second values are predetermined values represented by α and

β

, respectively, where β

>

α

, and

the score is interpolated between the first and second predetermined values when θ

₀<

F(ν

_i, I)<

θ

₁ by a transfer function made of at least two parabolic functions.

7. The method of claim 6, wherein the transfer function includes at least a₀[F(ν

_i, I)]²−

b₀F(ν

_i, I)+c₀ when θ

₀<

F(ν

_i, I)<

θ

′ and

(a₁[F(ν

_i,

     1)]²−

b₁F(ν

_i, I)+c₁ when θ

″

<

F(ν

_i, I)<

θ

₁, wherein a₀, a₁, b₀, b₁, c₀ and c₁ are constants and θ

′ and

θ

″

are fixed threshold values intermediate of θ

₁ and θ

₂.

8. The method of claim 7, wherein θ

′ and

θ

″

are equal.

9. The method of claim 4, wherein:

the dot product is represented by F(ν

_i, I), where ν

_i is a vector projected onto the current window I,

the first and second thresholds are represented by θ

₀ and θ

₁, respectively, where θ

₁>

θ

₀,

the first and second values are determined by parabolic transfer functions, and

the score is interpolated between the first and second values when θ

₀<

F(ν

_i, I)<

θ

₁ by a transfer function made of one or more parabolic functions.

10. The method of claim 1, wherein said evaluating comprises, for a window, rotating the stored data with respect to data of the image about an axis perpendicular to a plane of the window by changing a parameter of the stored data.

11. A method comprising:

establishing boundaries of windows within an image;

evaluating data of the image within individual windows with respect to stored data of a set of features of the particular type of object, by using a process that includes rotating the stored data with respect to data of the image about an axis perpendicular to a plane of the window by changing a parameter of the stored data, and

assigning scores to the individual windows that represent a likelihood of the presence of at least a portion of the object in the individual windows.

12. The method of claim 11, wherein the evaluation of data of the image within individual windows with respect to the stored feature set data additionally comprises:

maintaining the stored data of a set of features, for individual ones of a plurality of types of the object, the different object types representing different rotational positions of the object about an axis extending through the object and parallel to the plane to the window, and

wherein rotation of the stored feature set with respect to the image includes rotating the stored data for individual ones of the plurality of object types with respect to the perpendicular axis.

13. An electronic image acquisition device comprising:

an input interface circuit to receive data of an output of an image sensor, the data representing an image; and

processor circuitry coupled to the input interface circuit to receive the data and to provide processed data of the image, wherein the processor circuitry is configured to detect a likelihood that an object of a particular type is present within the image by processing that includes;

defining a plurality of windows within an image; and

assigning scores to individual windows of the plurality of windows by

evaluating data of the image within the individual windows with respect to stored data of a set of features of a particular type of object,

for each individual window, determining relationships between a result of said evaluating and a plurality of different thresholds, and

assigning a score to each individual window based on the determined relationships for said window.

14. The electronic image acquisition device of claim 13, wherein said assigning scores comprises:

for each individual window, comparing the result of said evaluating against a first threshold and a second threshold;

if the result is above a higher of the first and second thresholds, assigning a first value to the score;

if the result is below a lower of the first and second thresholds, assigning a second value to the score; and

if the result is between the first and second thresholds, assigning to the score a value interpolated between the first and second values.

15. The device of claim 13, further comprising said image sensor.

16. The device of claim 15, wherein said image sensor is a two-dimensional image sensor.

17. The device of claim 15, further comprising:

an optical system that projects an image of an object scene outside of the device onto the sensor.

18. The device of claim 13, wherein assigning to the score a value interpolated between the first and second values comprises performing interpolation between the first and second values non-linearly by a transfer function formed of at least two parabolic functions.

19. The device of claim 13, wherein the processing of the image processing circuitry to evaluate the likelihood of the presence of the set of features in the individual windows includes comparing a dot product of a vector of the image projected onto a current one of the windows with a vector of the current window.

20. The device of claim 13, wherein said evaluating comprises, for a window, rotating the stored data with respect to data of the image about an axis perpendicular to a plane of the window by changing a parameter of the stored data.

21. An electronic image acquisition device comprising:

an input interface circuit to receive data of an output of an image sensor, the data representing an image; and

processor circuitry coupled to the input interface circuit to receive the data and to provide processed data of the image, wherein the processor circuitry is configured to detect a likelihood that an object of a particular type is present within the image by performing operations including;

establishing boundaries of windows within the image,

evaluating data of the image within individual windows with respect to stored data of a set of features of the particular type of object by a process that includes rotating the stored data with respect to the data of the image about an axis perpendicular to a plane of a window by changing a parameter of the stored data, and

assigning scores to the individual windows that represent a likelihood of the presence of at least a portion of the object in the individual windows.

22. The device of claim 21, further comprising said image sensor.

23. The device of claim 22, wherein said image sensor is a two-dimensional image sensor.

24. The device of claim 22, further comprising:

an optical system that projects an image of an object scene outside of the device onto the sensor.

25. The device of claim 21, wherein said operations further include:

maintaining the stored data of a set of features, for individual ones of a plurality of types of the object, the different object types representing different rotational positions of the object about an axis extending through the object and parallel to the plane to the window, and

wherein rotation of the stored data with respect to the image includes rotating the stored data for individual ones of the plurality of object types with respect to the perpendicular axis.