Projecting visual aspects into a vector space

• US 10,628,481 B2
• Filed: 11/17/2016
• Issued: 04/21/2020
• Est. Priority Date: 11/17/2016
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Kiapour, Mohammadhadi, Piramuthu, Robinson
• Primary Examiner(s): Thai, Hanh B
• Application Number:
  US15/353,900
• Publication Number:
  US 20180137142A1
• Time in Patent Office: 1,251 Days
• Field of Search: 707706, 707737
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects

Abstract

In various example embodiments, a system and method for projecting visual aspects into a vector space are presented. A query that includes visual data is received by the system from a client device. A visual aspect indicated in the visual data is analyzed. One or more symbols that correspond to the analyzed visual aspect is generated by the system. The analyzed visual aspect is projected into a vector space using the one or more symbols. A group of projections are identified, the group of projections being within a predetermined distance from the projected visual aspect in the vector space. An interface that depicts the further visual aspects is generated. The interface is displayed on the client device.

Claims

1. A method comprising:

receiving a query that includes visual data from a client device;

analyzing a visual aspect indicated in the visual data;

generating one or more symbols that correspond to the analyzed visual aspect;

projecting, using one or more processors, the analyzed visual aspect into a vector space using the one or more symbols;

automatically identifying a group of projections that are within a predetermined distance from the projected visual aspect in the vector space, the group of projections being projections of further visual aspects indicated in further visual data;

determining that the visual aspect and the further visual aspects share a common characteristic;

in response to the determining, generating a user interface that displays the visual data and the further visual data, and depicts a tag indicating the common characteristic; and

causing display of the generated user interface on the client device.

2. The method of claim 1, wherein:

the visual data includes an image of an item; and

the analyzing includes extracting the visual aspect from a section of the image of the item.

3. The method of claim 1, further comprising:

receiving a description that corresponds to the visual data; and

parsing the description to identify a keyword describing the visual aspect indicated in the visual data.

4. The method of claim 3, wherein the generating the one or more symbols is based on the keyword parsed from the description.

5. The method of claim 1, further comprising:

receiving the further visual data from a further client device; and

storing the further visual data in a database.

6. The method of claim 1, wherein:

the further visual aspects are visual aspects of a group of items; and

the further visual data includes images of the group of items.

7. The method of claim 6, wherein:

the further visual data includes an item listing of an item from the group of items and

wherein the user interface is generated to display a link that is selectable to cause display of the item listing.

8. The method of claim 1, wherein the identifying the group of projections that are within a predetermined distance from the projected visual aspect in the vector space includes:

determining a position of the projected visual aspect in the vector space; and

determining that the group of projections are within the predetermined distance from the position of the projected visual aspect.

9. The method of claim 8, further comprising causing display of a user interface on the client device, the user interface prompting for a label of the tag.

10. The method of claim 1, further comprising:

generating a mapping based on data that indicates a pre-determined relationship between one or more further symbols and one or more further visual aspects, wherein the generating the one or more symbols is based on the generated mapping.

11. A system comprising:

one or more hardware processors; and

a memory storing executable instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising;

receiving a query that includes visual data from a client device;

analyzing a visual aspect indicated in the visual data;

generating one or more symbols that correspond to the analyzed visual aspect;

projecting the analyzed visual aspect into a vector space using the one or more symbols;

automatically identifying a group of projections that are within a predetermined distance from the projected visual aspect in the vector space, the group of projections being projections of further visual aspects indicated in further visual data;

determining that the visual aspect and the further visual aspects share a common characteristic;

in response to the determining, generating a user interface that depicts a tag indicating the common characteristic; and

causing display of the generated user interface on the client device.

12. The system of claim 11, wherein:

the visual data includes an image of an item; and

the analyzing includes extracting the visual aspect from a section of the image of the item.

13. The system of claim 11, wherein the operations further comprise:

receiving a description that corresponds to the visual data; and

parsing the description to identify a keyword describing the visual aspect indicated in the visual data.

14. The system of claim 13, wherein the generating the one or more symbols is based on the keyword parsed from the description.

15. The system of claim 11, wherein:

the further visual aspects are visual aspects of a group of items; and

the further visual data includes images of the group of items.

16. The system of claim 15, wherein:

the further visual data includes an item listing of an item from the group of items; and

generating the user interface includes generating a link that is selectable to view the item listing.

17. The system of claim 11, wherein the identifying the group of projections that are within a predetermined distance from the projected visual aspect in the vector space includes:

determining a position of the projected visual aspect in the vector space; and

determining that the group of projections are within the predetermined distance from the position of the projected visual aspect.

18. A non-transitory machine-readable medium storing instruction that, when executed by one or more hardware processors of a machine, cause the machine to perform operations comprising:

receiving a query that includes visual data from a client device;

analyzing a visual aspect indicated in the visual data;

generating one or more symbols that correspond to the analyzed visual aspect;

projecting the analyzed visual aspect into a vector space using the one or more symbols;

automatically identifying a group of projections that are within a predetermined distance from the projected visual aspect in the vector space, the group of projections being projections of further visual aspects indicated in further visual data;

determining that the visual aspect and the further visual aspects share a common characteristic;

in response to the determining, generating a user interface that displays the visual data, the further visual data, and depicts a tag indicating the common characteristic; and

causing display of the generated user interface on the client device.