Image processing apparatus and method, and computer-readable storage medium

• US 9,053,388 B2
• Filed: 10/08/2010
• Issued: 06/09/2015
• Est. Priority Date: 10/19/2009
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ito, Yoshinori, Kato, Masami, Yamamoto, Takahisa
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US12/901,364
• Publication Number:
  US 20110091113A1
• Time in Patent Office: 1,705 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

An image processing apparatus classifies a variation of a target object included in an image from a specific state as one of a plurality of types of attributes, and holds, for each variation attribute, a correction parameter for spatial transformation that corrects the target object to the specific state. The image processing apparatus generates an input image vector by vectorizing at least a partial region of the input image, and determines a variation attribute by detecting a variation of the target object from the specific state in the input image. Then, the image processing apparatus generates a transformed image vector by performing the spatial transformation on the input image vector, using a correction parameter selected based on the determined variation attribute from among the correction parameters held for respective variation attributes.

Claims

1. An image processing apparatus comprising:

a holding unit configured to hold a plurality of spatial transformation matrices for spatial transformation, and a projection matrix for projection operation;

a vectorizing unit configured to generate an input image vector by vectorizing at least a partial region of an input image;

a determination unit configured to determine a variation attribute by detecting a variation of a target object from a specific state in the input image; and

a generation unit configured to select one of the plurality of spatial transformation matrices based on the variation attribute determined by the determination unit, and to generate a transformed image vector by performing the spatial transformation on the input image vector using the selected spatial transformation matrix and to generate a projection vector by performing the projection operation on the transformed vector, using the projection matrix,

wherein in the spatial transformation matrices,

rows correspond to elements of the transformed image vector,

elements of the rows correspond to elements of the input image vector, and

in each row, only elements that are referenced when calculating an element value corresponding to the transformed image vector have a value indicating a reference ratio, and the other elements all have a value of 0.

9. The apparatus according to claim 1, further comprising:

a registration unit in which the projection vector generated by the generation unit is registered;

a calculation unit configured to calculate similarity between a projection vector generated from a new input image by the generation unit and the projection vector registered in the registration unit; and

an identification unit configured to identify whether the new input image includes the target object based on the similarity calculated by the calculation unit.

11. A non-transitory computer-readable storage medium storing a program for causing a computer to function as the units of the image processing apparatus according to claim 1.

2. An image processing apparatus comprising:

a holding unit configured to hold a plurality of synthesized projection matrices in which each of a plurality of spatial transformation matrices for each attribute and a normal state projection matrix for projection operation have been synthesized;

a vectorizing unit configured to generate an input image vector by vectorizing at least a partial region of an input image;

a determination unit configured to determine a variation attribute by detecting a variation of a target object from a specific state in the input image; and

a generation unit configured to select one of the plurality of synthesized projection matrices based on the variation attribute determined by the determination unit, and to generate a projection vector by performing the spatial transformation and the projection operation on the input image vector, using the selected synthesized projection matrix.

3. An image processing apparatus comprising:

a holding unit configured to hold a spatial transformation matrix for spatial transformation;

a vectorizing unit configured to generate an input image vector by vectorizing at least a partial region of an input image; and

a generation unit configured to generate a transformed image vector by performing the spatial transformation, using the spatial transformation matrix,

wherein in the spatial transformation matrix,

rows correspond to elements of the transformed image vector,

elements of the rows correspond to elements of the input image vector, and

in each row, only elements that are referenced when calculating an element value corresponding to the transformed image vector have a value indicating a reference ratio, and the other elements all have a value of 0.

4. The apparatus according to claim 3,

wherein the spatial transformation matrix is a deformation matrix that causes a shape in a varied state to be deformed to a shape in the specific state.

5. The apparatus according to claim 4,

wherein the reference ratio of the deformation matrix is set based on a relationship between pixels of an image corresponding to a fluctuated state and a coordinate position in an image corresponding to the specific state that have been obtained through learning using a plurality of pairs of an image corresponding to the varied state and an image corresponding to the specific state.

6. The apparatus according to claim 3,

wherein the spatial transformation matrix is a filter matrix that subjects the input image vector to linear spatial filter processing.

7. The apparatus according to claim 6,

wherein in the filter matrix,

in each row, in an element corresponding to the elements of the input image vector on which a filter kernel for a linear spatial filter processing operates, a coefficient corresponding to the filter kernel is set as a value indicating the reference ratio.

8. The apparatus according to claim 3,

wherein the spatial transformation matrix is a luminance distribution correction matrix that corrects a luminance distribution of the input image vector.

10. An image processing method performed by an image processing apparatus including a holding unit configured to hold a plurality of spatial transformation matrices for spatial transformation and a projection matrix for projection operation, the method comprising the steps of:

generating an input image vector by vectorizing at least a partial region of an input image;

determining a variation attribute by detecting a variation of a target object from a specific state in the input image; and

selecting one of the plurality of spatial transformation matrices based on the variation attribute determined in the determination step, and generating a transformed image vector by performing the spatial transformation on the input image vector using the selected spatial transformation matrix and generating a projection vector by performing the projection operation on the transformed vector, using the projection matrix,

wherein in the spatial transformation matrices,

rows correspond to elements of the transformed image vector,

elements of the rows correspond to elements of the input image vector, and

in each row, only elements that are referenced when calculating an element value corresponding to the transformed image vector have a value indicating a reference ratio, and the other elements all have a value of 0.

12. An image processing apparatus comprising:

a holding unit configured to hold a deformation matrix so as to transform a deformed image into a non-deformed image;

a vectorizing unit configured to generate an input image vector by vectorizing at least a partial region of an input image; and

a generation unit configured to generate a transformed image vector by transforming the input image vector using the held deformation matrix,

wherein the deformation matrix includes a reference ratio of an element in the deformed image when calculating an element value corresponding to the element in the deformed image.

13. The apparatus according to claim 12, wherein the holding unit further holds a projection matrix for projection operation.

14. The apparatus according to claim 13, wherein the generation unit generates the projection vector by projecting the input image vector using a synthesized projection matrix which is generated by synthesizing the deformation matrix and the projection matrix by a matrix product operation.

15. An image processing method performed by an image processing apparatus including a holding unit configured to hold a plurality of synthesized projection matrices in which each of a plurality of spatial transformation matrices for each attribute and a normal state projection matrix for projection operation have been synthesized, said method comprising the steps of:

generating an input image vector by vectorizing at least a partial region of an input image;

determining a variation attribute by detecting a variation of a target object from a specific state in the input image; and

selecting one of the plurality of synthesized projection matrices based on the variation attribute determined in the determining step, and generating a projection vector by performing the spatial transformation and the projection operation on the input image vector, using the selected synthesized projection matrix.

16. An image processing method performed by an image processing apparatus including a holding unit configured to hold a spatial transformation matrix for spatial transformation, said method comprising the steps of:

generating an input image vector by vectorizing at least a partial region of an input image; and

generating a transformed image vector by performing the spatial transformation, using the spatial transformation matrix,

wherein in the spatial transformation matrix,

rows correspond to elements of the transformed image vector,

elements of the rows correspond to elements of the input image vector, and

in each row, only elements that are referenced when calculating an element value corresponding to the transformed image vector have a value indicating a reference ratio, and the other elements all have a value of 0.

17. An image processing method performed by an image processing apparatus including a holding unit configured to hold a deformation matrix so as to transform a deformed image into a non-deformed image, said method comprising:

generating an input image vector by vectorizing at least a partial region of an input image; and

generating a transformed image vector by transforming the input image vector using the held deformation matrix,

wherein the deformation matrix includes a reference ratio of an element in the deformed image when calculating an element value corresponding to the element in the deformed image.