System and method for search with the aid of images associated with product categories

• US 9,817,901 B1
• Filed: 10/18/2016
• Issued: 11/14/2017
• Est. Priority Date: 06/28/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: Khafizov, Farid, Khafizova, Margarita
• Primary Examiner(s): Alam, Shahid
• Application Number:
  US15/296,672
• Time in Patent Office: 392 Days
• Field of Search: 707769
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0627 using item specifications
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof

Abstract

The present application describes performing a user initiated search query comprising receiving user input comprising description details of at least one desired object, retrieving a plurality of objects from a database sharing one or more of the description details of the user input, retrieving an image of the at least one desired object based on one or more of the plurality of objects, generating a contour of the image and comparing the generated contour with other related contours of other images stored in the database, displaying all of the available contours of all of the images that match the generated contour, receiving a selection of one of the available contours from the user and performing the search query based on the user selected contour.

Claims

1. A method, comprising:

receiving, via a receiver, user input comprising description details and image information of a desired object;

retrieving a plurality of objects from a database sharing one or more of the description details of the user input;

retrieving an image of the desired object based on the image information and determining a contour of the desired object from the retrieved image;

constructing a tree data structure based on the description details of the plurality of objects, the tree data structure comprising one or more attributes related to each of the plurality of retrieved objects;

mapping the determined contour from the image of the desired object to an image of a predetermined contour shape, from among a plurality of predetermined contour shapes stored in the database, as a category contour shape;

determining, via the processor, images of other retrieved objects that have a contour therein that is most closely matched by the image of the category contour shape from among the plurality of predetermined contour shapes, and displaying the images of the other retrieved objects having contours that are most closely matched by the image of the category contour shape; and

performing a search query based on a user selected image from among the displayed images.

2. The method of claim 1, wherein the image of the category contour shape comprises a contour shape that does not exactly match the contour of the desired object but that is determined to most closely match the contour of the desired object from among the plurality of predetermined contour shapes.

3. The method of claim 1, wherein the user initiated search query is performed via a user interface.

4. The method of claim 1, wherein the user input is at least one of text, audio and a digital image.

5. The method of claim 3, wherein the user input is a description of a desired product.

6. The method of claim 1, further comprising:

displaying categories of at least one value attribute of a level of the tree data structure that does not include images;

determining whether additional levels of the tree data structure are available that the user has not yet viewed; and

if so, displaying additional categories of the at least one value attribute of another level of the tree data structure that does not include images.

7. The method of claim 6, wherein the categories of the at least one value attribute and the additional categories of the at least one value attribute include at least one numeric predefined attribute value comprising at least one of a product price range, product warranty period, product length, and product weight.

8. An apparatus, comprising:

a receiver configured to receive user input comprising description details and image information of a desired object; and

a processor configured to

retrieve a plurality of objects from a database sharing one or more of the description details of the user input,

retrieve an image of the desired object based on the image information and determine a contour of the desired object from the retrieved image,

construct a tree data structure based on the description details of the plurality of objects, the tree data structure comprising one or more attributes related to each of the plurality of objects retrieved,

map the determined contour from the image of the desired object to an image of a predetermined contour shape, from among a plurality of predetermined contour shapes stored in the database, as a category contour shape;

determine images of other retrieved objects that have a contour therein that is most closely matched by the image of the category contour shape from among the plurality of predetermined contour shapes, and display the images of the other retrieved objects having contours that are most closely matched by the image of the category contour shape; and

perform a search query based on a user selected image from among the displayed images.

9. The apparatus of claim 8, wherein the image of the category contour shape comprises a contour shape that does not exactly match the contour of the desired object but that is determined to most closely match the contour of the desired object from among the plurality of predetermined contour shapes.

10. The apparatus of claim 8, wherein the user initiated search query is performed via a user interface.

11. The apparatus of claim 8, wherein the user input is at least one of text, audio and a digital image.

12. The apparatus of claim 11, wherein the user input is a description of a desired product.

13. The apparatus of claim 8, wherein the processor is further configured to display categories of at least one value attribute of a level of the tree data structure that does not include images, determine whether additional levels of the tree data structure are available that the user has not yet viewed, and, if so, to display additional categories of the at least one value attribute of another level of the tree data structure that does not include images.

14. The apparatus of claim 13, wherein the categories of the at least one value attribute and the additional categories of the at least one value attribute include at least one numeric predefined attribute value comprising at least one of a product price range, product warranty period, product length, and product weight.

15. A non-transitory computer readable storage medium configured to store instructions that when executed cause a processor to perform a method comprising:

receiving user input comprising description details and image information of a desired object;

retrieving a plurality of objects from a database sharing one or more of the description details of the user input;

retrieving an image of the desired object based on the image information and determining a contour of the desired object from the retrieved image;

constructing a tree data structure based on the description details of the plurality of objects, the tree data structure comprising one or more attributes related to each of the plurality of objects retrieved;

mapping the determined contour from the image of the desired object to an image of a predetermined contour shape, from among a plurality of predetermined contour shapes stored in the database, as a category contour shape;

determining, via the processor, images of other retrieved objects that have a contour therein that is most closely matched by the image of the category contour shape from among the plurality of predetermined contour shapes, and displaying the images of the other retrieved objects having contours that are most closely matched by the image of the category contour shape; and

performing a search query based on a user selected image from among the displayed images.

16. The non-transitory computer readable storage medium of claim 15, wherein the image of the category contour shape comprises a contour shape that does not exactly match the contour of the desired object but that is determined to most closely match the contour of the desired object from among the plurality of predetermined contour shapes.

17. The non-transitory computer readable storage medium of claim 15, wherein the user initiated search query is performed via a user interface.

18. The non-transitory computer readable storage medium of claim 15, wherein the user input is at least one of text, audio and a digital image.

19. The non-transitory computer readable storage medium of claim 18, wherein the user input is a description of a desired product.

20. The non-transitory computer readable storage medium of claim 15, wherein the method further comprises:

displaying categories of at least one value attribute of a level of the tree data structure that does not include images;

determining whether additional levels of the tree data structure are available that the user has not yet viewed; and

if so, displaying additional categories of the at least one value attribute of another level of the tree data structure that does not include images.