Adaptive search window control for visual search

• US 9,569,695 B2
• Filed: 04/24/2013
• Issued: 02/14/2017
• Est. Priority Date: 04/24/2012
• Status: Active Grant

• Current Assignee: STMicroelectronics International N.V. (STMicroelectronics NV)
• Original Assignee: STMicroelectronics SRL (STMicroelectronics NV)
• Inventors: Pasteris, Paolo, Pau, Danilo Pietro, Falchetto, Mirko
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US13/869,656
• Publication Number:
  US 20130279762A1
• Time in Patent Office: 1,392 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 30/18143 Extracting features based on salient regional features, e.g. scale invariant feature transform [SIFT] keypoints
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape

Abstract

Image-processing apparatus and methods to adaptively control a size and/or location of a visual search window used for feature matching in a machine-vision system are described. A search window controller may receive motion vector data and image recognition rate data, and compute a search window size and/or search window location based on the received data. The computed search window size may be a portion of an image frame. The motion vector data and image recognition rate data may be computed from one or more images in a video image sequence. By adaptively controlling search window size and location, an appreciable reduction in data processing burden for feature matching may be achieved.

Claims

1. An image processing system, comprising:

one or more memories; and

image processing circuitry, which, in operation;

controls a search window location of a search window associated with an image frame based on a motion vector associated with a sub-region of a previously processed image frame; and

controls a size and shape of the search window in response to a determined recognition rate, wherein the image processing circuitry, in operation, controls horizontal and vertical dimensions of the search window based on the motion vector associated with the sub-region of the previously processed image frame, according to;

2. The image processing system of claim 1, wherein the image processing circuitry comprises a field-programmable gate array.

3. The image processing system of claim 1 wherein the determined recognition rate is based on feature matching.

4. The image processing system of claim 1 wherein the image processing circuitry, in operation, encodes received video data and computes the motion vector.

5. The image processing system of claim 4 wherein the image processing circuitry, in operation, encodes video data according to a standardized format.

6. The image processing system of claim 5, wherein the standardized format comprises a MPEG or H264 format.

7. The image processing system of claim 1 wherein the image processing circuitry, in operation:

retrieves feature data from the one or more memories based upon the size of the search window; and

matches feature data for at least one feature of the image frame associated with the window against the feature data retrieved from the buffer.

8. The image processing system of claim 7 wherein the image processing circuitry, in operation, outputs optical flow data based upon matching results of feature data.

9. The image processing system of claim 7, comprising:

an integrated circuit including the one or more memories and the image processing circuitry, wherein the one or more memories include a buffer, which, in operation, stores feature data.

10. A method, comprising:

controlling, using image processing circuitry, a search window location of a search window associated with an image frame based on a motion vector associated with a sub-region of a previously processed image frame; and

controlling, using the image processing circuitry, a size and shape of the search window in response to a determined recognition rate, wherein the controlling the size and shape of the search window comprises controlling horizontal and vertical dimensions of the search window based on the motion vector associated with the sub-region of the previously processed image frame, according to;

11. The method of claim 10 wherein the image processing circuitry comprises a field-programmable gate array.

12. The method of claim 10, comprising generating, using the image processing circuitry, the determined recognition rate based on feature matching.

13. The method of claim 10, comprising:

encoding, received video data; and

computing the motion vector data.

14. The method of claim 13 wherein the encoding comprises encoding the received video data according to a standardized format.

15. The method of claim 14 wherein the standardized format comprises a MPEG or H264 format.

16. The method of claim 10, comprising:

retrieving feature data from a buffer based upon the size of the search window; and

matching feature data for at least one feature of the image frame associated with the window against the feature data retrieved from the buffer.

17. The method of claim 16, comprising outputting optical flow data based upon matching results of feature data.

18. A system, comprising

an image capture apparatus;

and

image processing circuitry communicatively coupled to the image capture apparatus, wherein the image processing circuitry, in operation;

controls a search window location of a search window associated with an image frame based on a motion vector associated with a sub-region of a previously processed image frame; and

controls a size and shape of the search window in response to a determined recognition rate, wherein the image processing circuitry, in operation, controls horizontal and vertical dimensions of the search window according to;

19. The system of claim 18 wherein the image processing circuitry comprises a field-programmable gate array.

20. The system of claim 18 wherein the determined recognition rate is based on feature matching.