System and method for search with the aid of images associated with product categories

• US 9,965,555 B1
• Filed: 11/01/2016
• Issued: 05/08/2018
• Est. Priority Date: 06/28/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Open Invention Network LLC
• Inventors: Khafizov, Farid, Khafizova, Margarita
• Primary Examiner(s): Alam, Shahid
• Application Number:
  US15/340,645
• Time in Patent Office: 553 Days
• Field of Search: 707769
• CPC Class Codes: G06F 16/2308 Concurrency control transaction processing G06F9/466
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0627 using item specifications
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof

Abstract

The present application describes performing a user initiated search query comprising receiving user input comprising description details of at least one desired object, retrieving a plurality of objects from a database sharing one or more of the description details of the user input, retrieving an image of the at least one desired object based on one or more of the plurality of objects, generating a contour of the image and comparing the generated contour with other related contours of other images stored in the database, displaying all of the available contours of all of the images that match the generated contour, receiving a selection of one of the available contours from the user and performing the search query based on the user selected contour.

Claims

1. A method, comprising:

receiving a contour of an image via a processor;

retrieving at least one additional contour of at least one additional image stored in a database;

determining whether the received contour of the image matches the at least one additional contour of the at least one additional image;

in response to determining that the received contour of the image is not a match with the at least one additional contour of the at least one additional image, rotating the at least one additional contour of the at least one additional image based on a rotation angle;

extracting a new contour from the rotated at least one additional contour; and

performing an image-based search using the new contour extracted from the rotated at least one additional contour, and displaying information about search results generated from the image-based search on a display device,

wherein extracting the new contour from the rotated at least one additional contour is performed based on pixels forming an edge portion of the new image, and the extracting comprises;

transforming the new contour and the at least one additional contour into binary images; and

resizing the new contour to match a size of the at least one additional contour.

2. The method of claim 1, further comprising receiving the image as user input.

3. The method of claim 1, further comprising calculating a distance transform function of the transformed binary images of the new contour and the at least one additional contour.

4. The method of claim 3, wherein the distance function is a sum of absolute values of coordinate differences for two pixels, one pixel being part of the new contour and the other pixel being part of the at least one additional contour.

5. The method of claim 3, wherein the distance function is a maximum of the absolute differences for x and y coordinates for two pixels, one pixel being part of the new contour and the other pixel being part of the at least one additional contour.

6. The method of claim 1, further comprising retrieving a plurality of images from the database that are related to the user'"'"'s input image.

7. The method of claim 1, further comprising determining whether the created contour matches the new contour from the rotated at least one additional contour, and in response to determining that the created contour matches, setting the new contour as a category contour image representative of the created contour, and performing the image based search to identify search results that are related to the category contour image.

8. An apparatus, comprising:

a processor configured to

receive a contour of an image;

retrieve at least one additional contour of at least one additional image stored in a database;

determine whether the received contour of the image matches the at least one additional contour of the at least one additional image;

in response to determining that the received contour of the image is not a match with the at least one additional contour of the at least one additional image, rotate the at least one additional contour of the at least one additional image based on a rotation angle;

extract a new contour from the rotated at least one additional contour; and

perform an image-based search using the new contour extracted from the rotated at least one additional contour, and display information about search results generated from the image-based search on a display device,

wherein the extraction of the new contour from the rotated at least one additional contour is performed based on pixels that form an edge portion of the new image, and the extraction comprises;

a transformation of the new contour and the at least one additional contour into binary images, and

a resize of the new contour to match a size of the at least one additional contour.

9. The apparatus of claim 8 further comprising a receiver configured to receive the image as user input.

10. The apparatus of claim 8, wherein the processor is further configured to calculate a distance transform function of the transformed binary images of the new contour and the at least one additional contour.

11. The apparatus of claim 10, wherein the distance function is a sum of absolute values of coordinate differences for two pixels, one pixel part of the new contour and the other pixel part of the at least one additional contour.

12. The apparatus of claim 10, wherein the distance function is a maximum of the absolute differences for x and y coordinates for two pixels, one pixel part of the new contour and the other pixel part of the at least one additional contour.

13. The apparatus of claim 8, wherein the processor is further configured to retrieve a plurality of images from the database that are related to the user'"'"'s input image.

14. A non-transitory computer readable storage medium configured to store instructions that when executed cause a processor to perform a method comprising:

receiving a contour of an image;

retrieving at least one additional contour of at least one additional image stored in a database;

determining whether the received contour of the image matches the at least one additional contour of the at least one additional image;

in response to determining that the received contour of the image is not a match with the at least one additional contour of the at least one additional image, rotating the at least one additional contour of the at least one additional image based on a rotation angle;

extracting a new contour from the rotated at least one additional contour; and

performing an image-based search using the new contour extracted from the rotated at least one additional contour, and displaying information about search results generated from the image-based search on a display device,

wherein extracting the new contour from the rotated at least one additional contour is performed based on pixels forming an edge portion of the new image, and the extracting comprises;

transforming the new contour and the at least one additional contour into binary images; and

resizing the new contour to match a size of the at least one additional contour.

15. The non-transitory computer readable storage medium of claim 14, further comprising receiving an image as user input.

16. The non-transitory computer readable storage medium of claim 14, further comprising calculating a distance transform function of the transformed binary images of the new contour and the at least one additional contour.

17. The non-transitory computer readable storage medium of claim 16, wherein the distance function is a sum of absolute values of coordinate differences for two pixels, one pixel being part of the new contour and the other pixel being part of the at least one additional contour.

18. The non-transitory computer readable storage medium of claim 17, wherein the distance function is a maximum of the absolute differences for x and y coordinates for two pixels, one pixel being part of the new contour and the other pixel being part of the at least one additional contour.