Interest point detection

• US 8,712,162 B2
• Filed: 06/15/2011
• Issued: 04/29/2014
• Est. Priority Date: 01/18/2011
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Kirsch, Graham
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/161,324
• Publication Number:
  US 20120183224A1
• Time in Patent Office: 1,049 Days
• Field of Search: 382/195, 382/264
• US Class Current: 382/195
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 10/449 Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

Interest points are markers anchored to a specific position in a digital image of an object. They are mathematically extracted in such a way that, in another image of the object, they will appear in the same position on the object, even though the object may be presented at a different position in the image, a different orientation, a different distance or under different lighting conditions. Methods are disclosed that are susceptible to implementation in hardware and corresponding hardware circuits are described.

Claims

1. A method of identifying candidate interest points in an image having rows and columns of image data, the method comprising:

receiving the rows of image data in series;

generating downscaled image data by blurring the image data received, wherein the downscaled image data represents the blurred data from a pattern of pixel locations in the received image data;

identifying candidate interest points at a first scale by Hessian-filtering the image data and detecting extrema in the Hessian-filtered data;

discarding the rows of image data in series;

identifying candidate interest points at a second scale by Hessian-filtering the downscaled image data and detecting extrema in the Hessian-filtered, downscaled data, wherein at least one of the rows of image data is discarded before all the rows of image data have been received;

generating twice downscaled image data by blurring the downscaled image data, wherein the twice downscaled image data represents the blurred data from a pattern of pixel locations in the once downscaled image data; and

detecting candidate interest points at a third scale by Hessian-filtering the twice downscaled image data and detecting extrema in the Hessian-filtered, twice downscaled data.

2. The method of claim 1 comprising discarding at least one of the rows of once downscaled image data before all the rows of once downscaled image data have been generated.

3. The method of claim 1 comprising, for m equals 2 to n, where n is an integer greater than 2:

generating m-times downscaled image data by blurring m−

1-times downscaled image data, wherein the m-times downscaled image data represents the blurred data from a pattern of pixel locations in the m−

1-times downscaled image data;

identifying candidate interest points at an (m+1)th scale by Hessian filtering the m-times downscaled image data and detecting extrema in the Hessian-filtered, m-times downscaled data; and

discarding the rows of m−

1-times downscaled image data in series.

4. The method of claim 3 wherein, for m equals 2 to n−

1, at least one of the rows of m−

1-times downscaled image data is discarded before all the rows of m−

1-times downscaled image data have been generated.

5. The method of claim 4 wherein the received image data is received into and discarded from a line buffer.

6. The method of claim 5 wherein the downscaled image data is also received into a line buffer.

7. The method of claim 6 wherein the received image data is received into and discarded from a line buffer and all the downscaled image data is also received into the same line buffer.

8. The method of claim 7 wherein identifying candidate interest points at a particular scale comprises applying at least three Hessian filters spaced in scale space to the image data and detecting extrema in the resulting Hessian-filtered data.

9. The method of claim 8 wherein identifying candidate interest points at a particular scale comprises applying more than three Hessian filters spaced in scale space to the image data and detecting extrema in the resulting Hessian-filtered data at more than one scale.

10. The method of claim 9 wherein the Hessian filters are equally spaced in scale space.

11. A circuit for identifying candidate interest points in an image having rows and columns of image data, the circuit comprising:

at least one input for receiving the rows of image data in series;

a line buffer for storing at most a subset of the rows of image data as they are received, whereby at least one of the rows of image data is to be discarded from the line buffer before all the rows of image data have been received;

a convolution engine adapted to convolve received image data with a blurring filter and to output blurred data from a pattern of pixel locations in the received image data as downscaled image data, and to convolve received image data with at least three Hessian filters of different scales and to output the Hessian filtered data;

an interest point identifier adapted to receive the Hessian filtered data from the convolution engine, to detect extrema in the Hessian-filtered data to identify candidate interest points and to output the position and scale of the candidate interest points so identified; and

sequencing circuitry adapted to sequence the operation of the circuit to pass received image data from the line buffer to the convolution engine to be convolved with both the blurring filter and the Hessian filters, to pass downscaled image data from the blurring filter back to the convolution engine to be convolved with the Hessian filters, and to discard the rows of received image data in series, whereby the interest point identifier identifies candidate interest points at a first scale in the Hessian-filtered received image data and at a second scale in the Hessian-filtered, downscaled data and outputs the position and scale of the candidate interest points so identified.

12. The circuit of claim 11 wherein the sequencing circuitry is adapted to sequence the operation of the circuit to pass downscaled image data from the line buffer to the convolution engine to be convolved with both the blurring filter and the Hessian filters, to pass twice downscaled image data from the blurring filter back to the convolution engine to be convolved with the Hessian filters, and to discard the rows of once downscaled image data in series, whereby the interest point identifier also identifies candidate interest points at a third scale in the Hessian-filtered, twice downscaled data and outputs the position and scale of the candidate interest points so identified.

13. The circuit of claim 11 wherein the sequencing circuitry is adapted to sequence the operation of the circuit, for m equals 2 to n, where n is an integer greater than 2, to pass m−

1-times downscaled image data from the line buffer to the convolution engine to be convolved with both the blurring filter and the Hessian filters, to pass m-times downscaled image data from the blurring filter back to the convolution engine to be convolved with the Hessian filters, and to discard the rows of m−

1-times downscaled image data in series, whereby the interest point identifier also identifies candidate interest points at an (m+1)th scale in the Hessian-filtered, m-times downscaled data and outputs the position and scale of the candidate interest points so identified.

14. The circuit of claim 13 wherein the sequencing circuitry is adapted to sequence the operation of the circuit to pass to pass twice downscaled image data, or m-times downscaled image data as the case may be, from the blurring filter back to the line buffer and then from the line buffer to the convolution engine to be convolved with the Hessian filters.

15. A method of downscaling and organising image data, the method comprising:

receiving image data organised into rows and columns;

storing the image data in a line buffer organised into rows and columns, wherein the rows of image data are stored in successive rows of the line buffer and, in each row, the image data is stored in successive columns;

generating downscaled image data by blurring the image data received, wherein the downscaled image data is organised into rows and columns and represents the blurred data from a pattern of pixel locations in the received image data;

storing the downscaled image data in the same line buffer, wherein the rows of the downscaled image data are stored in successive rows of the line buffer, at least one of which also stores received image data and, in each row, the downscaled image data is stored in successive unused columns;

generating twice downscaled image data by blurring the downscaled image data, wherein the twice downscaled image data is organised into rows and columns and represents the blurred data from a pattern of pixel locations in the once downscaled image data; and

storing the twice downscaled image data in the same line buffer, wherein the rows of the twice downscaled image data are stored in successive rows of the line buffer, at least one of which also stores received and once downscaled image data and, in each row, the twice downscaled image data is stored in successive unused columns of the line buffer.

16. The method of claim 15 comprising, for m equals 2 to n, where n is an integer greater than 2:

generating m-times downscaled image data by blurring m−

1-times downscaled image data, wherein the m-times downscaled image data represents the blurred data from a pattern of pixel locations in the m−

1-times downscaled image data;

storing the m-times downscaled image data into the same line buffer, wherein the rows of the m-times downscaled image data are stored in successive rows of the line buffer, at least one of which also stores received and m−

1 times downscaled image data and any and all intermediate downscaled image data, and, in each row, the m-times downscaled image data is stored in successive unused columns.

17. The method claim 16 wherein more than one row of the line buffer stores both received image data and downscaled image data.

18. The method of claim 16 wherein more than one row of the line buffer stores received image data, downscaled image data and twice downscaled image data.