Image processing device, image processing method, program, and integrated circuit

• US 8,520,950 B2
• Filed: 03/18/2011
• Issued: 08/27/2013
• Est. Priority Date: 05/24/2010
• Status: Expired due to Fees

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Ishii, Yasunori, Hiramoto, Masao
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): GILLIARD, DELOMIA L
• Application Number:
  US13/386,500
• Publication Number:
  US 20120121189A1
• Time in Patent Office: 893 Days
• Field of Search: None
• US Class Current: 382/197
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: H04N 2101/00 Still video cameras

Abstract

The image processing device includes: a storage unit (211) holding intensity gradient vectors Vr, position vectors Rr, and voting vectors Ur of a reference image; an intensity gradient vector calculation unit (212) which calculates intensity gradient vectors Vs of a search image; and a position determination unit (213) which determines a position of the reference image in the search image. The position determination unit (213) includes: a sampling unit (214) which thins out a part of the intensity gradient vectors Vs and/or the voting vectors Ur; an origin position estimation unit (215) which locates voting vectors Ur at each starting position of intensity gradient vectors Vs and estimates ending positions of the voting vectors Ur as candidate points; and a re-verification unit (216) which locates the position vectors Rr at each candidate point and determines a candidate point having most intensity gradient vectors Vs at ending positions of the position vectors Rr as an origin position.

Claims

1. An image processing device that determines a position of a reference image in a search image, said image processing device comprising:

a storage unit configured to store a plurality of intensity gradient vectors Vr, a plurality of position vectors Rr, and a plurality of voting vectors Ur in association with one another, the intensity gradient vectors Vr each indicating a gradient of an intensity value between pixels included in the reference image, the position vectors Rr being expressed by a first two-dimensional coordinate system and each extending from an origin set on a plane of the reference image to a starting position of a corresponding one of the intensity gradient vectors Vr, and the voting vectors Ur each being expressed by a corresponding second two-dimensional coordinate having a reference axis in a direction of a corresponding one of the intensity gradient vectors Vr and each extending from a starting position of the corresponding one of the intensity gradient vectors Vr to the origin and;

an intensity gradient vector calculation unit configured to calculate a plurality of intensity gradient vectors Vs each indicating a gradient of an intensity value between pixels included in the search image; and

a position determination unit configured to determine a position of the reference image in the search image, by determining, on a plane of the search image, a position corresponding to the origin on the plane of the reference image, based on (a) the intensity gradient vectors Vs which are calculated by said intensity gradient vector calculation unit and (b) the position vectors Rr and the voting vectors Ur which are stored in said storage unit,

wherein said position determination unit includes;

a sampling unit configured to extract target intensity gradient vectors Vs and target voting vectors Ur, by thinning out a part of at least one of (a) the intensity gradient vectors Vs calculated by said intensity gradient vector calculation unit and (b) the voting vectors Ur stored in said storage unit;

an origin position estimation unit configured to (i) locate the target voting vectors Ur at each of starting positions of the target intensity gradient vectors Vs, wherein the target voting vectors Ur are rotated to match the reference axis to a corresponding one of the intensity gradient vectors Vs, and (ii) estimate ending positions of the target voting vectors Ur as candidate points for the position corresponding to the origin; and

a re-verification unit configured to (i) locate the position vectors Rr on each of the candidate points estimated by said origin position estimation unit, and (ii) determine a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the intensity gradient vectors Vs each existing at an ending position of a corresponding one of the position vectors Rr.

2. The image processing device according to claim 1,

wherein said re-verification unit is further configured to (i) calculate, for each of the candidate points estimated by said origin position estimation unit, a rotation angle between (a) an intensity gradient vector Vs existing at a starting position of a corresponding one of the voting vectors Ur indicating the each of the candidate points and (b) an intensity gradient vector Vr stored in said storage unit in association with the corresponding one of the voting vectors Ur, and (ii) determine a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the intensity gradient vectors Vs existing at an ending position of a corresponding one of the position vectors Rr which are rotated only by the rotation angle.

3. The image processing device according to claim 1,

wherein said re-verification unit is further configured to (i) count, for each of the candidate points estimated by said origin position estimation unit, from among intensity gradient vectors Vs each existing at an ending position of a corresponding one of the position vectors Rr, intensity gradient vectors Vs each of which has a substantially same size or a substantially same direction as a size or direction of an intensity gradient vector Vr stored in said storage unit in association with the corresponding one of the position vectors Rr, and (ii) determine a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the counted intensity gradient vectors Vs.

4. The image processing device according to claim 1,

wherein said position determination unit further includes a feature calculation unit configured to calculate a feature that is a difference value between (a) a pixel value of a certain pixel and (b) a mean value of pixel values of a plurality of neighboring pixels of the certain pixel, and

said re-verification unit is further configured to (i) count, for each of the candidate points estimated by said origin position estimation unit, from among pixels each existing at an ending position of a corresponding one of the position vectors Rr, pixels each having a substantially same feature as a feature of a corresponding pixel in the reference image, and (ii) determine a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the counted pixels.

5. The image processing device according to claim 1,

wherein said origin position estimation unit is configured to estimate, as the candidate points, only points each of which is indicated by ending positions of the target voting vectors whose number is equal to or more than a predetermined value.

6. The image processing device according to claim 1,

wherein said origin position estimation unit is further configured to locate the target voting vectors Ur at each of starting positions of the target intensity gradient vectors Vs, the target voting vectors Ur having been scaled by a predetermined scaling factor.

7. An image processing method of causing an image processing device to determine a position of a reference image in a search image, the image processing device including a storage unit configured to store a plurality of intensity gradient vectors Vr, a plurality of position vectors Rr, and a plurality of voting vectors Ur in association with one another, the intensity gradient vectors Vr each indicating a gradient of an intensity value between pixels included in the reference image, the position vectors Rr being expressed by a first two-dimensional coordinate system and each extending from an origin set on a plane of the reference image to a starting position of a corresponding one of the intensity gradient vectors Vr, the voting vectors Ur each being expressed by a corresponding second two-dimensional coordinate having a reference axis in a direction of a corresponding one of the intensity gradient vectors Vr and each extending from a starting position of the corresponding one of the intensity gradient vectors Vr to the origin, and said image processing method comprising:

calculating a plurality of intensity gradient vectors Vs each indicating a gradient of an intensity value between pixels included in the search image; and

determining a position of the reference image in the search image, by determining, on a plane of the search image, a position corresponding to the origin on the plane of the reference image, based on (a) the intensity gradient vectors Vs which are calculated in said calculating and (b) the position vectors Rr and the voting vectors Ur which are stored in the storage unit,

wherein said determining includes;

extracting target intensity gradient vectors Vs and target voting vectors Ur, by thinning out a part of at least one of (a) the intensity gradient vectors Vs calculated in said calculating and (b) the voting vectors Ur stored in the storage unit;

estimating ending positions of the target voting vectors Ur as candidate points for the position corresponding to the origin, by locating the target voting vectors Ur at each of starting positions of the target intensity gradient vectors Vs, wherein the target voting vectors Ur are rotated to match the reference axis to a corresponding one of the intensity gradient vectors Vs; and

determining a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the intensity gradient vectors Vs each existing at an ending position of a corresponding one of the position vectors Rr, by locating the position vectors Rr on each of the candidate points estimated in said estimating.

8. A non-transitory computer-readable recording medium for use in a computer to determine a position of a reference image in a search image, the computer including a storage unit configured to store a plurality of intensity gradient vectors Vr, a plurality of position vectors Rr, and a plurality of voting vectors Ur in association with one another, the intensity gradient vectors Vr each indicating a gradient of an intensity value between pixels included in the reference image, the position vectors Rr being expressed by a first two-dimensional coordinate system and each extending from an origin set on a plane of the reference image to a starting position of a corresponding one of the intensity gradient vectors Vr, the voting vectors Ur each being expressed by a corresponding second two-dimensional coordinate having a reference axis in a direction of a corresponding one of the intensity gradient vectors Vr and each extending from a starting position of the corresponding one of the intensity gradient vectors Vr to the origin, and said recording medium having a computer program recorded thereon for causing the computer to execute:

calculating a plurality of intensity gradient vectors Vs each indicating a gradient of an intensity value between pixels included in the search image; and

determining a position of the reference image in the search image, by determining, on a plane of the search image, a position corresponding to the origin on the plane of the reference image, based on (a) the intensity gradient vectors Vs which are calculated in said calculating and (b) the position vectors Rr and the voting vectors Ur which are stored in the storage unit,

wherein said determining includes;

extracting target intensity gradient vectors Vs and target voting vectors Ur, by thinning out a part of at least one of (a) the intensity gradient vectors Vs calculated in said calculating and (b) the voting vectors Ur stored in the storage unit;

estimating ending positions of the target voting vectors Ur as candidate points for the position corresponding to the origin, by locating the target voting vectors Ur at each of starting positions of the target intensity gradient vectors Vs, wherein the target voting vectors Ur are rotated to match the reference axis to a corresponding one of the intensity gradient vectors Vs; and

determining a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the intensity gradient vectors Vs each existing at an ending position of a corresponding one of the position vectors Rr, by locating the position vectors Rr on each of the candidate points estimated in said estimating.

9. An integral circuit which determines a position of a reference image in a search image, said integral circuit being embedded in an image processing device including a storage unit configured to store a plurality of intensity gradient vectors Vr, a plurality of position vectors Rr, and a plurality of voting vectors Ur in association with one another, the intensity gradient vectors Vr each indicating a gradient of an intensity value between pixels included in the reference image, the position vectors Rr being expressed by a first two-dimensional coordinate system and each extending from an origin set on a plane of the reference image to a starting position of a corresponding one of the intensity gradient vectors Vr, the voting vectors Ur each being expressed by a corresponding second two-dimensional coordinate having a reference axis in a direction of a corresponding one of the intensity gradient vectors Vr and each extending from a starting position of the corresponding one of the intensity gradient vectors Vr to the origin, and said integral circuit comprising:

an intensity gradient vector calculation unit configured to calculate a plurality of intensity gradient vectors Vs each indicating a gradient of an intensity value between pixels included in the search image; and

a position determination unit configured to determine a position of the reference image in the search image, by determining, on a plane of the search image, a position corresponding to the origin on the plane of the reference image, based on (a) the intensity gradient vectors Vs which are calculated by said intensity gradient vector calculation unit and (b) the position vectors Rr and the voting vectors Ur which are stored in said storage unit,

wherein said position determination unit includes;

a sampling unit configured to extract target intensity gradient vectors Vs and target voting vectors Ur, by thinning out a part of at least one of (a) the intensity gradient vectors Vs calculated by said intensity gradient vector calculation unit and (b) the voting vectors Ur stored in said storage unit;

an origin position estimation unit configured to (i) locate the target voting vectors Ur at each of starting positions of the target intensity gradient vectors Vs, wherein the target voting vectors Ur are rotated to match the reference axis to a corresponding one of the intensity gradient vectors Vs, and (ii) estimate ending positions of the target voting vectors Ur as candidate points for the position corresponding to the origin; and

a re-verification unit configured to (i) locate the position vectors Rr on each of the candidate points estimated by said origin position estimation unit, and (ii) determine a candidate point among the candidate points as the position corresponding to the origin, the candidate point having a largest number of the intensity gradient vectors Vs each existing at an ending position of a corresponding one of the position vectors Rr.