Robust interest point detector and descriptor

• US 8,670,619 B2
• Filed: 04/17/2012
• Issued: 03/11/2014
• Est. Priority Date: 04/28/2006
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Eidgenoessische Technische Hochschule Zurich, KU Leuven Research & Development, Toyota Motor Europe SA (Toyota Motor Corporation)
• Inventors: Funayama, Ryuji, Yanagihara, Hiromichi, Van Gool, Luc, Tuytelaars, Tinne, Bay, Herbert
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/448,562
• Publication Number:
  US 20120201466A1
• Time in Patent Office: 693 Days
• Field of Search: 382/190, 382/260
• US Class Current: 382/190
• CPC Class Codes: G06V 10/36 Applying a local operator, i.e. means to operate on image points situated in the vicinity of a given point; Non-linear local filtering operations, e.g. median filtering
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A method for operating on images is described for interest point detection and/or description working under different scales and with different rotations, e.g. for scale-invariant and rotation-invariant interest point detection and/or description.

Claims

1. A method for determining an interest point in an image having a plurality of pixels suitable for working at different scales and/or rotations, the method comprising:

filtering the image using at least three digital filters, and selecting an interest point based on determining a measure resulting from application of the at least three digital filters, the measure being a non-linear combination of the outputs of the at least three digital filters, the measure capturing variations of an image parameter in more than one dimension or direction, the at least three digital filters being a combination of box filters, a first digital filter having a central lobe of one binary polarity, two zero crossing and two outer lobes of the opposite binary polarity and having a spatial extent greater than one pixel and a second digital filter wherein the central pixel has a zero value and there are two positive lobes along a diagonal and there are two negative lobes along the other diagonal, and a third digital filter that is a rotated version of the first filter.

2. The method of claim 1 wherein application of at least one digital filter to the image is performed with integral images.

3. The method of claim 1, wherein the combination of box filters approximates derivatives of a smoothing filter in more than one direction.

4. The method of claim 1, further comprising providing a plurality of filtered images at different scales using at least one the digital filters.

5. The method of claim 1 wherein the measure is the determinant of a Hessian matrix constructed from the results of applying at least one of the filters.

6. The method of claim 1, wherein at least one of the digital filters is derived from the second order derivative of a smoothing filter.

7. The method, according to claim 1, wherein at least one of the digital filters is applied at different scalings to the image to generate a plurality of filtered images.

8. The method of claim 1 wherein an interest point is determined as a local extreme value of the measure within a neighbourhood including a region of a filtered image.

9. The method of claim 8, wherein the neighbourhood is a space defined by at least three of a plurality of filtered images logically arranged in an image pyramid.

10. The method of claim 1 further comprising deriving a descriptor of an interest point in an image having a plurality of pixels, the interest point having a location in the image and an orientation, the method comprising:

identifying a region in a neighbourhood around the interest point aligned with the orientation of the interest point, the neighbourhood comprising a set of pixels;

examining tiles of the region, and for each tile generating a contrast related response using at least one digital filter, summing response values from application of the at least one digital filter in at least two orthogonal directions to generate summed values, and generating a multidimensional descriptor having first elements, each first element being based on the summed values.

11. The method of claim 10 wherein the descriptor the sign of the Laplacian at the interest point.

12. The method of claim 1 further comprising deriving a descriptor of an interest point in an image having a plurality of pixels, the interest point having a location in the image and an orientation, and a neighbourhood having been defined around the interest point aligned with the orientation of the interest point, the neighbourhood comprising a set of pixels;

the method comprising;

inspecting contrasts in the neighbourhood of the interest point using at least one digital filter,

generating a multidimensional descriptor based on the results of the application of the at least one digital filter and absolute values of these results.

13. The method of claim 1 further comprising assigning the orientation of an interest point in an image having a plurality of pixels, the interest point having a location, the method comprising:

identifying a region enclosing the interest point,

determining an orientation for the interest point by;

examining a plurality of tiles of the region, each tile comprising a plurality of pixels, determining for each tile filtered values related to contrast in two directions to thereby determine for that tile an orientation and a magnitude for that orientation, and assigning an orientation to the interest point by selecting the determined orientation with largest magnitude.