Interactive interfaces for generating annotation information

• US 10,635,280 B2
• Filed: 08/09/2019
• Issued: 04/28/2020
• Est. Priority Date: 11/28/2016
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Natesh, Aishwarya, Zamalieva, Daniya
• Primary Examiner(s): Motsinger, Sean T
• Application Number:
  US16/536,910
• Publication Number:
  US 20190361595A1
• Time in Patent Office: 263 Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Approaches provide for an interactive interface for determining annotation information that can be used to recommend content, improve user interaction and satisfaction, as well as improve various image recognition approaches. For example, a user can be presented an interactive interface, such as a game. The interactive interface can include colored graphical elements, where each graphical element can be associated with and display a visual feature of an item. A user can be provided a task or goal, such as to select, move, or otherwise arrange graphical elements based on a visual feature of the elements. The user interaction can generate annotation information that can be utilized to determine related content as well as improve various image recognition approaches.

Claims

1. A computer-implemented method, comprising:

receiving image data that includes a representation of an object;

dividing at least a portion of the image data into a plurality of regions;

determining at least one descriptor for each of the plurality of regions;

comparing the at least one descriptor for each of the plurality of regions with at least one stored descriptor of a set of stored descriptors, the at least one stored descriptor associated with a color or a texture;

generating an interface that includes a plurality of graphical elements, each graphical element of the plurality of graphical elements comprising at least one of a respective color or texture associated with at least one stored descriptor of the set of stored descriptors, each graphical element of the plurality of graphical elements presented separate from the image data, wherein at least a subset of the set of stored descriptors is associated with a respective product in an electronic catalog of products; and

in response to receiving a user interaction of at least one graphical element of the graphical elements, generating annotation information for a product associated with the at least one graphical element.

2. The computer-implemented method of claim 1, further comprising:

generating the interface that includes the plurality of graphical elements in response to receiving the image data.

3. The computer-implemented method of claim 2, further comprising:

analyzing the image data to determine the at least one descriptor for each of the plurality of regions, the at least one descriptor for each of the plurality of regions associated with a color, a pattern, or a texture of at least part of the image data.

4. The computer-implemented method of claim 3, further comprising:

generating a comparison score based on comparing the at least one descriptor for each of the plurality of regions with the at least one stored descriptor.

5. The computer-implemented method of claim 1, further comprising:

generating a comparison score associated with a comparison between a descriptor for the at least one graphical element and a stored descriptor, and

generating annotation information when the comparison score satisfies a threshold.

6. The computer-implemented method of claim 1, wherein the user interaction comprises a user selection of a graphical element having a color, a texture, or a pattern that is associated with a corresponding color, texture, or pattern of a product.

7. The computer-implemented method of claim 1, wherein generating the interface occurs following at least one of:

an execution of an image recognition algorithm,

a predetermined time interval,

completion of an electronic purchase transaction, or

a user'"'"'s placement of a product in a virtual shopping cart in an electronic marketplace.

8. The computer-implemented method of claim 1, wherein the user interaction of at least one graphical element comprises selecting the at least one graphical element or moving the at least one graphical element.

9. A system, comprising:

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the system to;

receive image data that includes a representation of an object;

divide at least a portion of the image data into a plurality of regions;

determine at least one descriptor for each of the plurality of regions;

compare the at least one descriptor for each of the plurality of regions with at least one stored descriptor of a set of stored descriptors, the at least one stored descriptor associated with a color or a texture;

generate an interface that includes a plurality of graphical elements, each graphical element of the plurality of graphical elements comprising at least one of a respective color or texture associated with at least one stored descriptor of the set of stored descriptors, wherein at least a subset of the set of stored descriptors is associated with a respective product in an electronic catalog of products;

receive a user interaction of at least one graphical element of the graphical elements; and

generate annotation information for a product associated with the at least one graphical element, based at least in part on the user interaction.

10. The system of claim 9, wherein the user interaction of at least one graphical element of the graphical elements comprises at least one of:

selecting the at least one graphical element,

selecting adjacent graphical elements, or

moving a graphical element.

11. The system of claim 9, wherein the instructions when executed cause the system to generate an interface in response to:

a completed execution of an image recognition algorithm;

a user'"'"'s reservation of an item for purchase in a virtual shopping cart;

arrival at a predetermined time interval;

or

a user'"'"'s completion of an electronic purchase.

12. The system of claim 11, wherein the instructions when executed further cause the system to:

analyze an image following the completed execution of an image recognition algorithm; and

determine at least one visual descriptor for at least one portion of the image, the at least one visual descriptor associated with a color, texture, or pattern of the at least one portion of the image.

13. The system of claim 12, wherein the instructions when executed further cause the system to:

compare the at least one visual descriptor for the at least one portion of the image with at least one stored visual descriptor;

determine a similarity score based on the step of comparing the at least one visual descriptor with the at least one stored visual descriptor;

determine whether the similarity score meets a predetermined selection criterion; and

generate the interface that includes the plurality of graphical elements using the at least one stored visual descriptor when the similarity score meets the predetermined selection criterion.

14. The system of claim 12, wherein the at least one visual descriptor for the at least one portion of the image is determined using a trained classifier.

15. The system of claim 9, wherein the instructions when executed further cause the system to:

evaluate the annotation information for the product associated with the at least one graphical element; and

identify other products that are associated with the annotation information for the product associated with the at least one graphical element.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to:

receive image data that includes a representation of an object;

divide at least a portion of the image data into a plurality of regions;

determine at least one descriptor for each of the plurality of regions;

compare the at least one descriptor for each of the plurality of regions with at least one stored descriptor of a set of stored descriptors, the at least one stored descriptor associated with a color or a texture;

generate a plurality of graphical elements, each graphical element of the plurality of graphical elements including at least one of a color, a pattern, or a texture associated with at least one stored descriptor of the set of stored descriptors, each graphical element of the plurality of graphical elements presented separate from the image data;

receive a user interaction of at least one graphical element of the plurality of graphical elements; and

generate annotation information for a product associated with the at least one graphical element, based at least in part on the user interaction.

17. The non-transitory computer-readable storage medium of claim 16, wherein the user interaction of at least one graphical element of the plurality of graphical elements comprises at least one of:

selecting the at least one graphical element,

selecting adjacent graphical elements,

selecting graphical elements having the same color and/or complementary colors, or

moving a graphical element.

18. The non-transitory computer-readable storage medium of claim 16, wherein the instructions when executed cause the computing system to generate a plurality of graphical elements in response to at least one of:

an unsuccessful execution of an image recognition algorithm;

an addition of a product to a virtual shopping cart in an electronic marketplace;

a predetermined time interval;

a completion of a product purchase in an electronic marketplace.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions when executed further cause the computing system to:

generate a descriptor associated with at least one of a color, a pattern, or a texture; and

generate a comparison scored based on a comparison between the descriptor associated with the color, the pattern, or the texture and a stored descriptor associated with a stored color, a stored pattern, or a stored texture.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions when executed further cause the computing system to generate a graphical element when the comparison score meets a predetermined selection criterion.