Interactive interfaces for generating annotation information
First Claim
1. A computing device comprising:
- at least one processor;
a camera; and
memory storing instructions that, when executed by the at least one processor, cause the computing device to perform operations, comprising;
obtain image data that includes a representation of an object using the camera;
analyze the image data to attempt to recognize the object;
determine an image recognition confidence score below a threshold confidence score, the image recognition confidence score indicating a likelihood of recognizing the object;
segment the image data into a plurality of regions;
analyze the plurality of regions to determine a color descriptor for each of the plurality of regions, the color descriptor for each of the plurality of regions describing a color represented in image data for a respective region;
compare respective color descriptors against a stored set of color descriptors to determine a plurality of visual similarity scores, each of the stored set of color descriptors associated with a patch of a product that is associated with color information;
determine a set of patches based at least in part on the plurality of visual similarity scores;
generate an interface that includes a plurality of graphical elements, each graphical element associated with a patch of the set of patches and displaying a color of its associated patch;
receive a user selection of adjacent graphical elements; and
generate color annotation information based at least in part on the adjacent graphical elements.
1 Assignment
0 Petitions
Accused Products
Abstract
Approaches provide for an interactive interface for determining annotation information that can be used to recommend content, improve user interaction and satisfaction, as well as improve various image recognition approaches. For example, a user can be presented an interactive interface, such as a game. The interactive interface can include colored graphical elements, where each graphical element can be associated with and display a visual feature of an item. A user can be provided a task or goal, such as to select, move, or otherwise arrange graphical elements based on a visual feature of the elements. The user interaction can generate annotation information that can be utilized to determine related content as well as improve various image recognition approaches.
-
Citations
20 Claims
-
1. A computing device comprising:
-
at least one processor; a camera; and memory storing instructions that, when executed by the at least one processor, cause the computing device to perform operations, comprising; obtain image data that includes a representation of an object using the camera; analyze the image data to attempt to recognize the object; determine an image recognition confidence score below a threshold confidence score, the image recognition confidence score indicating a likelihood of recognizing the object; segment the image data into a plurality of regions; analyze the plurality of regions to determine a color descriptor for each of the plurality of regions, the color descriptor for each of the plurality of regions describing a color represented in image data for a respective region; compare respective color descriptors against a stored set of color descriptors to determine a plurality of visual similarity scores, each of the stored set of color descriptors associated with a patch of a product that is associated with color information; determine a set of patches based at least in part on the plurality of visual similarity scores; generate an interface that includes a plurality of graphical elements, each graphical element associated with a patch of the set of patches and displaying a color of its associated patch; receive a user selection of adjacent graphical elements; and generate color annotation information based at least in part on the adjacent graphical elements. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method, comprising:
-
obtaining image data that includes a representation of an object; determining a visual descriptor for an image region of the image data; associating the visual descriptor with a patch of color of a plurality of patches of color based at least in part on a similarity between the visual descriptor and the patch of color, each of the patches of color associated with an area of a product of an electronic catalog of products; generating an interface that includes a plurality of graphical elements, one of the plurality of graphical elements associated with the patch of color, at least a portion of the remaining plurality of graphical elements associated with one of the plurality of patches of color and displaying a color of its associated patch of color; and in response to receiving a user interaction of the graphical elements, generating annotation information. - View Dependent Claims (6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A computing system, comprising:
-
at least one processor; and memory including instructions that, when executed by the at least one processor, cause the computing system to; obtain image data that includes a representation of an object; determine a visual descriptor for an image region of the image data; associate the visual descriptor with a patch of color of a plurality of patches of color based at least in part on a similarity between the visual descriptor and the patch of color, each of the patches of color associated with an area of a product of an electronic catalog of products; generate an interface that includes a plurality of graphical elements, each element of the plurality of graphical elements associated with one of the plurality of patches of color and displaying a color of its associated patch of color; and in response to receiving a selection of adjacent graphical elements, generate annotation information. - View Dependent Claims (17, 18, 19, 20)
-
Specification