OBJECT RECOGNITION OF FEATURE-SPARSE OR TEXTURE-LIMITED SUBJECT MATTER

• US 20170255648A1
• Filed: 05/22/2017
• Published: 09/07/2017
• Est. Priority Date: 03/13/2014
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: DUBE, SIMANT, HSIAO, EDWARD
• Granted Patent: US 10,650,040 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects

Abstract

An object recognition system can be adapted to recognize subject matter having very few features or limited or no texture. A feature-sparse or texture-limited object can be recognized by complementing local features and/or texture features with color, region-based, shape-based, three-dimensional (3D), global, and/or composite features. Machine learning algorithms can be used to classify such objects, and image matching and verification can be adapted to the classification. Further, multiple modes of input can be integrated at various stages of the object recognition processing pipeline. These multi-modal inputs can include user feedback, additional images representing different perspectives of the object or specific regions of the object including a logo or text corresponding to the object, user behavior data, location, among others.

Claims

1. A system, comprising:

at least one processor;

memory including instructions that, when executed by the at least one processor, cause the system to;

obtain a request to identify an object represented in an image;

determine a first image attribute and a second image attribute associated with the object, the first image attribute associated with a first pixel region that includes a representation of the object, the second image attribute associated with a second pixel region outside the first pixel region of a second size, an area of the second pixel region greater than an area of the first pixel region;

fail to identify the object based on the first image attribute;

determine a classification of the object based at least in part upon the second image attribute; and

determine a database object matching the object based at least in part upon the classification.

2. The system of claim 1, wherein the instructions when executed further cause the system to:

obtain additional input data corresponding to a user associated with the request to identify the object, and

determine the database object matching the object based at least in part upon the additional input data, and wherein the additional input data includes subsequent image data including at least one of a different perspective of the object, a logo corresponding to the object, text corresponding to the object, user behavior data relating to the object, or a location of a computing device that captured the image data.

3. The system of claim 1, wherein the first image attribute is a local feature and the second image attribute is a non-local feature.

4. The system of claim 1, wherein the first image attribute is different from the second image attribute.

5. The system of claim 1, wherein the second image attribute is one of a color feature attribute, a region-based feature attribute, a shape-based feature attribute, a global feature attribute, a three-dimensional (3D) feature attribute, or a composite feature attribute.

6. The system of claim 1, wherein the instructions when executed further cause the system to:

use a machine learning algorithm with the second image attribute to determine the classification.

7. The system of claim 1, wherein the instructions when executed further cause the system to:

verify that the database object matches the object based at least in part on a geometric verification of the second image attribute.

8. A computer-implemented method, comprising:

obtaining a request to identify an object represented in an image;

determining a first image attribute and a second image attribute associated with the object, the first image attribute associated with a first pixel region that includes a representation of the object, the second image attribute associated with a second pixel region outside the first pixel region of a second size, an area of the second pixel region greater than an area of the first pixel region;

failing to identify the object based on the first image attribute;

determining a classification of the object based at least in part upon the second image attribute; and

determining a database object matching the object based at least in part upon the classification.

9. The computer-implemented method of claim 8, further comprising:

analyzing a database that includes stored images to determine a stored image associated with the classification;

determining a second image attribute for the stored image;

determining a similarity measure between the second image attribute and the second image attribute for the stored image; and

determining whether the stored image matches the image based at least in part on the similarity measure.

10. The computer-implemented method of claim 8, further comprising:

obtaining information corresponding to the database object; and

presenting the information.

11. The computer-implemented method of claim 10, wherein the information overlays a live view of the object.

12. The computer-implemented method of claim 11, wherein the information includes an interactive element for changing an appearance of the object.

13. The computer-implemented method of claim 8, further comprising:

obtaining additional input data corresponding to a user associated with the request to identify the object, and

determining the database object matching the object based at least in part upon the additional input data, and wherein the additional input data includes subsequent image data including at least one of a different perspective of the object, a logo corresponding to the object, text corresponding to the object, user behavior data relating to the object, or a location of a computing device that captured the image data.

14. The computer-implemented method of claim 8, further comprising:

using a machine learning algorithm with the second image attribute to determine the classification.

15. The computer-implemented method of claim 8, further comprising:

verifying that the database object matches the object based at least in part on a geometric verification of the second image attribute.

16. A non-transitory computer-readable storage medium storing instructions, the instructions, which when executed by at least one processor of a computing system, cause the computing system to:

obtain a request to identify an object represented in an image;

determine a first image attribute and a second image attribute associated with the object, the first image attribute associated with a first pixel region that includes a representation of the object, the second image attribute associated with a second pixel region outside the first pixel region of a second size, an area of the second pixel region greater than an area of the first pixel region;

fail to identify the object based on the first image attribute;

determine a classification of the object based at least in part upon the second image attribute; and

determine a database object matching the object based at least in part upon the classification.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing system to:

obtain additional input data corresponding to a user associated with the request to identify the object, and

determine the database object matching the object based at least in part upon the additional input data, and wherein the additional input data includes subsequent image data including at least one of a different perspective of the object, a logo corresponding to the object, text corresponding to the object, user behavior data relating to the object, or a location of a computing device that captured the image data.

18. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing system to:

use a machine learning algorithm with the second image attribute to determine the classification.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing system to:

verify that the database object matches the object based at least in part on a geometric verification of the second image attribute.

20. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing system to:

analyze a database that includes stored images to determine a stored image associated with the classification;

determine a second image attribute for the stored image;

determine a similarity measure between the second image attribute and the second image attribute for the stored image;

determine whether the stored image matches the image based at least in part on the similarity measure;

obtain information corresponding to the database object; and

present the information.