Image object retrieval

• US 9,870,517 B2
• Filed: 12/08/2011
• Issued: 01/16/2018
• Est. Priority Date: 12/08/2011
• Status: Expired due to Fees

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Excalibur IP, LLC (Acacia Research Corporation)
• Inventors: van Zwol, Roelof, Pueyo, Lluis Garcia
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/315,210
• Publication Number:
  US 20130148903A1
• Time in Patent Office: 2,231 Days
• Field of Search: 382199, 382201, 382205, 382218-221, 382305, 707E17019-E1703
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

Subject matter disclosed herein relates to image object recognition or retrieval.

Claims

1. A method of matching sample images with reference images utilizing one or more special-purpose computing devices including one or more processors coupled to one or more memory devices, comprising:

accessing computer instructions from the one or more memory devices coupled to the one or more special-purpose computing devices for execution on the one or more processors of the one or more special-purpose computing devices;

executing the accessed computer instructions utilizing the one or more computing devices; and

storing, in the one or more memory devices coupled to the one or more special-purpose computing devices, any results of having executed the accessed computer instructions on the one or more processors of the one or more special-purpose computing devices, wherein executing the accessed computer instructions comprises;

superimposing, responsive to receipt of one or more signals from a user interface, a shape around an object of interest in at least a portion of an image;

quantizing image descriptors for the at least a portion of the image, the quantized image descriptors including one or more sample image interest points and one or more interest points neighboring the one or more sample image interest points;

determining whether two or more image interest points belong to the same or related quantization class, based, at least in part, on testing for neighborhood similarity of two or more sample image interest points;

quantizing edges between the one or more sample image interest points and the one or more interest points neighboring the one or more sample image interest points to form an edge index for the at least a portion of the image, the quantized image descriptors including a geometrical distance and an angle between the one or more sample image interest points and the one or more interest points neighboring the one or more sample image interest points;

searching for matches between the edge index for the at least a portion of the image and an edge index of reference images based, at least in part, on the tested neighborhood similarity; and

providing one or more of the reference images to an Internet browser responsive to searching for the one or more matches.

2. The method of claim 1, further comprising determining offset values which comprise the geometrical distance between the one or more sample image interest points and the one or more interest points neighboring the one or more sample image interest points.

3. The method of claim 2, wherein the one or more sample image interest points and the one or more interest points neighboring the one or more sample image interest points include a scale, and wherein

forming one or more edge signatures between the one or more sample image interest points and the one or more points neighboring the one or more sample image interest points based at least in part on the scale.

4. The method of claim 1, wherein magnitudes of image pixel characteristics of reference images are stored in the edge index of reference images.

5. The method of claim 4, wherein quantization identifications (IDs) of the one or more interest points neighboring one or more reference image interest points are stored in the edge index for the at least a portion of the image and the edge index of reference images.

6. The method of claim 4, further comprising:

in response to a search query to search for an object of the at least a portion of the image, forming N signatures for a sample image interest point and N points neighboring the sample image interest point; and

comparing the N signatures to components stored in the edge index to determine a match between the object of the at least a portion of the image and an object in the reference images.

7. The method of claim 1, further comprising quantizing image descriptors that are at least approximately scale invariant.

8. The method of claim 7, wherein the quantized image descriptors describe interest points of an object, wherein relative positions of the interest points are substantially constant with respect to varying perspectives of the object.

18. The method of claim 6, wherein the shape for superimposing around an object of interest in the at least a portion of the image comprises a geometrical shape.

9. An apparatus to match sample images with reference images utilizing one or more special-purpose computing devices including one or more processors coupled to one or more memory devices, comprising:

a computing device to;

access computer instructions from the one or more memory devices coupled to the one or more special-purpose computing devices for execution on the one or more processors of the one or more special-purpose computing devices;

execute the accessed computer instructions utilizing the one or more computing devices; and

store, in the one or more memory devices coupled to the one or more special-purpose computing devices, any results of having executed the accessed computer instructions on the one or more processors of the one or more special-purpose computing devices, wherein executing the accessed computer instructions comprise instructions to;

execute computer instructions to superimpose, responsive to receipt of one or more signals from a user interface, a shape around an object of interest in at least a portion of an image;

quantize image descriptors for at least a portion of an image, the quantized image descriptors to include one or more sample image interest points and one or more interest points to neighbor the one or more sample image interest points;

determine whether two or more image interest points belong to the same or related quantization class, based, at least in part, on testing for neighborhood similarity of two or more sample image interest points;

quantize edges between the one or more sample image interest points and the one or more interest points to neighbor the one or more sample image interest points to form an edge index for the at least the portion of the image, the quantized image descriptors to include a geometrical distance and an angle between the one or more sample image interest points and the one or more interest points to neighbor the one or more sample image interest points;

search for matches between the edge index for the at least a portion of the image and an edge index of reference images based, at least in part, on the tested neighborhood similarity; and

provide one or more of the reference images to an Internet browser responsive to searching for the one or more matches.

10. The apparatus of claim 9, wherein magnitudes of image pixel characteristics of reference images are stored in the edge index.

11. The apparatus of claim 10, wherein quantization identifications (IDs) of the one or more interest points to neighbor one or more interest points of the reference images.

12. The apparatus of claim 10, further comprising:

in response to a search query to search for an object of the at least a portion of the image, the computing device to form N signatures for a sample image interest point and N interest points to neighbor the sample image interest point; and

to compare the N signatures to components to be stored in the edge index to determine a match between the object of the at least a portion of the image and an object in the reference images.

13. The apparatus of claim 9, further comprising quantizing image descriptors that are to be at least approximately scale invariant.

14. An apparatus to match sample images with reference images utilizing one or more special-purpose computing devices including one or more processors coupled to one or more memory devices, comprising:

means for accessing computer instructions from the one or more memory devices coupled to the one or more special-purpose computing devices for execution on the one or more processors of the one or more special-purpose computing devices;

means for executing the accessed computer instructions utilizing the one or more computing devices; and

means for storing, in the one or more memory devices coupled to the one or more special-purpose computing devices, any results of having executed the accessed computer instructions on the one or more processors of the one or more special-purpose computing devices, wherein executing the accessed computer instructions comprises;

means for superimposing, responsive to receipt of one or more signals from a user interface, a shape around an object of interest in at least a portion of an image;

means for quantizing image descriptors for the at least a portion of the image, the quantized image descriptors to include interest point neighborhood state parameters;

means for determining whether two or more image interest points belong to the same or related quantization class, based, at least in part, on testing for neighborhood similarity of two or more sample image interest points;

means for quantizing edges between one or more sample image interest points and one or more interest points neighboring the one or more sample image interest points to be stored in an edge index for the at least a portion of the image, the quantized image descriptors including a geometrical distance and an angle between the one or more sample image interest points and the one or more interest points neighboring the one or more sample image interest points;

means for searching for matches between the edge index for the at least a portion of the image and an edge index of reference images based, at least in part, on the tested neighborhood similarity; and

means for providing one or more of the reference images to an Internet browser responsive to searching for the one or more matches.

15. The apparatus of claim 14, wherein components to be stored in the edge index of reference images to comprise quantization identifications (IDs) of one or more interest points neighboring one or more interest points in the reference images.

16. The apparatus of claim 14, wherein the means for quantizing comprises means for forming N signatures for a sample image interest point and N interest points neighboring the sample image interest point;

and

the means for searching comprising means for comparing the N signatures to components to be stored in the edge index for the image to determine a match between an object of the at least a portion of the image and an object in the reference images.

17. The apparatus of claim 14, further comprising quantizing image descriptors that are to be at least approximately scale invariant.