Adaptive interest rate control for visual search

• US 9,600,744 B2
• Filed: 04/24/2013
• Issued: 03/21/2017
• Est. Priority Date: 04/24/2012
• Status: Active Grant

• Current Assignee: STMicroelectronics International N.V. (STMicroelectronics NV)
• Original Assignee: STMicroelectronics SRL (STMicroelectronics NV)
• Inventors: Pau, Danilo Pietro, Falchetto, Mirko
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US13/869,652
• Publication Number:
  US 20130279813A1
• Time in Patent Office: 1,427 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 30/18143 Extracting features based on salient regional features, e.g. scale invariant feature transform [SIFT] keypoints
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape

Abstract

Image-processing apparatus and methods to adaptively vary an interest point threshold value and control a number of interest points identified in an image frame are described. Sub-regions of an image frame may be processed in a sequence, and an interest point threshold value calculated for each sub-region. The calculated value of the interest point threshold may depend upon pre-selected values and values determined from the processing of one or more prior sub-regions. By using adaptive thresholding, a number of interest points detected for each frame in a sequence of image frames may remain substantially constant, even though objects within the frames may vary appreciably.

Claims

1. An image processing system, comprising:

one or more memories configured to store image data; and

image processing circuitry, which, in operation;

identifies one or more interest points in a first sub-region of image data associated with an image frame based upon an interest point threshold value;

determines the interest point threshold value based at least in part upon a number of interest points identified in a previous sub-region of image data for the image frame; and

processes image data associated with the image frame based on the identified interest points, wherein the image processing circuitry is configured to;

determine the interest point threshold value based in part upon a recognition rate;

generate descriptor data for at least one identified interest point; and

evaluate matches between features of two or more images based at least in part on the descriptor data, wherein the image processing circuitry is configured to determine the interest point threshold value based in part upon a target number of interest points calculated for the previous sub-region.

2. The image processing system of claim 1 wherein the image processing circuitry comprises a field-programmable gate array.

3. The image processing system of claim 1 wherein the image processing circuitry includes interest point detection circuitry and rate control circuitry and the interest point detection circuitry is configured to receive the interest point threshold value from the rate control circuitry and to communicate a number of identified interest points in the first sub-region to the rate control circuitry.

4. The image processing system of claim 1 wherein the image processing circuitry is configured to determine the interest point threshold value based in part upon a target number of interest points preselected for the image frame.

5. The image processing system of claim 1 wherein the target number of interest points calculated for the previous sub-region is based at least in part on a total number of interest points pre-selected for the image frame and a total number of interest points detected for all prior processed sub-regions of the image frame.

6. The image processing system of claim 1 wherein the recognition rate is a value calculated for a previous sub-region.

7. The image processing system of claim 1 wherein the recognition rate is a cumulative rate determined based on all prior processed sub-regions of the image frame.

8. The image processing system of claim 1 wherein the image processing circuitry is configured to determine the recognition rate.

9. The image processing system of claim 1 wherein the image processing circuitry is configured to limit an amount of interest point data.

10. The image processing system of claim 9 wherein the image processing circuitry is configured to selectively discard interest point data based upon a target number of interest points for an image sub-region.

11. A method, comprising:

identifying, using image processing circuitry, one or more interest points in a first sub-region of image data associated with an image frame based upon an interest point threshold value;

determining, using the image processing circuitry, the interest point threshold value based at least in part upon a number of interest points identified in a previous sub-region of image data for the image frame, wherein the interest point threshold value is based in part upon a recognition rate;

processing, using the image processing circuitry, image data associated with the image frame based on the identified interest points;

generating, using the image processing circuitry, descriptor data for at least one identified interest point; and

evaluating, using the image processing circuitry, a quality of matching between features of two or more images based at least in part on the descriptor data, wherein the interest point threshold value is based in part upon a target number of interest points calculated for the previous sub-region.

12. The method of claim 11, comprising determining the recognition rate.

13. The method of claim 11, comprising limiting an amount of interest point data.

14. A non-transitory, computer-readable medium having contents which cause one or more image processing devices to perform a method, the method comprising:

identifying, one or more interest points in a first sub-region of image data associated with an image frame based upon an interest point threshold value;

determining the interest point threshold value based at least in part upon a number of interest points identified in a previous sub-region of image data for the image frame, wherein the interest point threshold value is based in part upon a recognition rate; and

processing image data associated with the image frame based on the identified interest points, wherein the method comprises;

generating descriptor data for at least one identified interest point; and

evaluating a quality of matching between features of two or more images based at least in part on the descriptor data, wherein the interest point threshold value is based in part upon a target number of interest points calculated for the previous sub-region.

15. The non-transitory, computer-readable medium of claim 14 wherein the method comprises determining the interest point threshold value based in part upon a target number of interest points pre-selected for the image frame.

16. The non-transitory, computer-readable medium of claim 14 wherein the recognition rate is a value calculated for a previous sub-region.