Image processing apparatus, photographic subject identifying method and program

• US 9,798,955 B2
• Filed: 12/25/2014
• Issued: 10/24/2017
• Est. Priority Date: 12/26/2013
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Bao, Ruihan, Higa, Kyota
• Primary Examiner(s): Mariam, Daniel
• Application Number:
  US15/100,694
• Publication Number:
  US 20160300122A1
• Time in Patent Office: 1,034 Days
• Field of Search: 382201, 382218, 382225
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

Image processing apparatus includes a local amount generation unit, a correspondence point calculation unit, a relative correspondence point information calculation unit, a correspondence point selecting unit and a decision unit. The local feature value generation unit calculates, for a first image, a set of information about first local feature values including a first feature point(s). The correspondence point calculation unit calculates, for a second image, as information about correspondence point(s), a correspondence relation between the first and second feature point(s) contained in a set of information about second local feature values calculated from the second image. The relative correspondence point information calculation unit calculates relationships of scales of feature points, as the information about the relative scale sizes of correspondence points, based on the set of information about the first local feature values and the set of information about the second local feature values on the information about correspondence point(s).

Claims

1. An image processing apparatus, comprising:

a first local feature value generation unit that detects one or more of first feature points from a first image, and calculates, from a region of a preset range encompassing each first feature point detected, a set of information about first local feature values corresponding to each first feature point;

a correspondence point calculation unit that calculates, as information about correspondence point(s), a correspondence relation between the first feature point(s) and a second feature point(s) contained in a set of information about second local feature values calculated from a second image;

a relative correspondence point scale size calculation unit that calculates, as information about relative scale sizes of correspondence points, a relationship between a scale(s) of the first feature point(s) and a scale(s) of the second feature point(s), based on the set of the information about the first local feature values, the set of the information about the second local feature values and the information about correspondence points;

a correspondence point selecting unit that, based on the information about the relative scale sizes of the correspondence points, performs clustering on at least one out of the first feature point(s) and the second feature point, and selects at least one feature point based on an outcome of the clustering; and

a decision unit that, based on the feature point(s) selected by the correspondence point selecting unit, compares the first image and the second image with each other, from cluster to cluster, and decides on possible photographic subject identity.

3. The image processing apparatus according to claim 1, wherein,

the relative correspondence point scale size calculation unit normalizes the information about the relative scale sizes of the correspondence points.

4. The image processing apparatus according to claim 1, wherein,

the correspondence point selecting unit performs clustering on the first feature point(s) based on the relative correspondence point information and on coordinate values of the first feature point(s).

5. The image processing apparatus according to claim 1, wherein,

the correspondence point selecting unit generates selecting information that correlates the information about the correspondence points of the feature points selected and one or more of cluster information with each other.

6. The image processing apparatus according to claim 1, further comprising:

a relative coordinate value calculation unit that selects a reference point from the second image, and calculates relationship between the first feature point(s) and the reference point(s), as relative coordinate information, based on the first feature point(s), second feature point(s) and the relative correspondence point information.

7. The image processing apparatus according to claim 1,

wherein,

the correspondence point selecting unit performs clustering on at least one out of the first feature point(s) and the second feature point(s), based on the relative correspondence point information and a preset range of the relative correspondence point information.

2. An image processing apparatus, comprising:

a first local feature value generation unit that detects one or more of first feature points from a first image, and calculates, from a region of a preset range encompassing each first feature point detected, a set of information about first local feature values corresponding to each first feature point;

a correspondence point calculation unit that calculates, as information about correspondence point(s), a correspondence relation between the first feature point(s) and a second feature point(s) contained in a set of information about second local feature values calculated from a second image;

a relative correspondence point orientation calculation unit that calculates, as information about relative orientations of the correspondence points, a relationship between an orientation of the first feature point(s) and an orientation of the second feature point(s), based on the set of the information about the first local feature values, the set of information about the second local feature values and the information about correspondence points;

a correspondence point selecting unit that, based on the information about relative orientations of the correspondence points, performs clustering on at least one out of the first feature point(s) and the second feature point(s), and selects at least one feature point based on an outcome of the clustering; and

a decision unit that, based on the feature point(s) selected by the correspondence point selecting unit, compares the first image and the second image with each other, from cluster to cluster, and decides on possible photographic subject identity.

8. A method for identifying a photographic subject, comprising:

detecting one or more of first local feature points from a first image, calculating, from a region of a preset range encompassing each first feature point detected, a set of information about first local feature values corresponding to each first feature point;

calculating, as information about correspondence points, a correspondence relation between the first feature point(s) and a second feature point(s) contained in a set of information about second local feature values calculated from a second image;

calculating, as information about relative scale sizes of correspondence points, a relationship between a scale of the first feature point(s) and a scale of the second feature point(s), based on the set of the information about first local feature values, the set of information about second local feature values and the information about correspondence points;

performing clustering on at least one out of the first feature point(s) and the second feature point(s), based on the information about the relative scale sizes of the correspondence points, and selecting at least one feature point based on an outcome of the clustering; and

comparing the first image and the second image with each other, based on the feature point(s) selected, from cluster to cluster, and deciding on possible photographic subject identity.

9. A method for identifying a photographic subject, comprising:

detecting one or more of first local feature points from a first image, calculating, from a region of a preset range encompassing each first feature point detected, a set of information about first local feature values corresponding to each first feature point;

calculating, as information about correspondence point(s), a correspondence relation between the first feature point(s) and a second feature point(s) contained in a set of information about second local feature values calculated from a second image;

calculating, as information about relative orientations of correspondence points, a relationship between an orientation of the first feature point(s) and an orientation of the second feature point(s), based on the set of information about the first local feature values, the set of information about the second local feature values and the information about the correspondence points;

performing clustering on at least one out of the first feature point(s) and the second feature point(s), based on the information about the relative orientations of the correspondence points, and selecting at least one feature point based on an outcome of the clustering; and

comparing the first image and the second image with each other, based on the feature point(s) selected, from cluster to cluster, and deciding on possible photographic subject identity.

10. A non-transitory computer readable storage medium storing a program that causes a computer controlling an image processing apparatus to perform the processing of:

detecting one or more of first feature points from a first image to calculate, from a region of a preset range encompassing each first feature point detected, a set of information about first local feature values corresponding to each first feature point;

calculating, as information about correspondence points, a correspondence relation between the first feature point(s) and a second feature point(s) contained in a set of information about second local feature values calculated from a second image;

calculating, as information about the relative scale sizes of the correspondence points, a relationship between a scale of the first feature point(s) and a scale of the second feature point(s), based on the set of information about the first local feature values, the set of the information about the second local feature values and the information about correspondence points;

performing clustering on at least one out of the first feature point(s) and the second feature point(s), based on the information about the relative scale sizes of the correspondence points, and selecting at least one feature point based on an outcome of the clustering; and

comparing the first image and the second image with each other, based on the feature point(s) selected, from cluster to cluster, deciding on possible photographic subject identity.