SYSTEM AND METHOD FOR USING IMAGE ANALYSIS AND SEARCH IN E-COMMERCE

• US 20080177640A1
• Filed: 11/07/2007
• Published: 07/24/2008
• Est. Priority Date: 05/09/2005
• Status: Abandoned Application

• Current Assignee: Google Inc. (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Shah, Munjal, Torresani, Lorenzo, Gokturk, Salih Burak, Vu, Diem, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar, Yang, Danny, Lin, Xiaofan, Sumengen, Baris, Dalal, Navneet
• Application Number:
  US11/936,694
• Publication Number:
  US 20080177640A1
• Time in Patent Office: Days
• US Class Current: 705/27
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers

Abstract

Embodiments described herein provide a system and method for providing merchandise items at a network site. According to an embodiment, an image of a merchandise item is obtained. The image is programmatically analyzed to determine information about the merchandise item. The information is used to generate a presentation that includes the merchandise item.

Claims

26. A method for providing a user-interface to offer merchandise items on a network site, the method comprising:

for each of a plurality of terminals that connect to the site across a network, displaying a first plurality of panels, wherein each panel includes at least (i) an image of a merchandise item for sale, and (ii) a link to a network location where the merchandise item is purchasable;

enabling a user of each of the plurality of terminals to select or specify, as a search criteria, at least one of (i) one of the plurality of panels, (ii) an image contained in the one of the plurality of panels, (iii) a portion of the image of the one of the plurality of panels, or (iv) a feature or characteristic of the image of one of the panels;

in response to receiving the search criteria, providing a result that identifies one or more panels or merchandise items.

27. The method of claim 26, wherein providing a result that identifies one or more panels includes providing a second plurality of panels that each include an image or information that satisfies the search criteria.

28. The method of claim 26, further comprising enabling a user to select a category of merchandise items, and wherein displaying a first plurality of panels includes selecting panels containing images of merchandise items that are identified as being part of the selected category or classification.

29. The method of claim 26, further comprising enabling a user to specify a text input as an additional or alternative search criteria.

30. The method of claim 26, wherein enabling the user of each of the plurality of terminals to select includes enabling the user to select, as the search criteria, any one or more of (i) any one of the plurality of panels, (ii) the portion of the image of the one of the plurality of panels, or (iii) the feature or characteristic of the image of one of the panels.

31. A system for providing merchandise items at a network site, the system comprising:

a collection of records, each record including (i) at least one of an image, or data representing an image, of a merchandise item, and (ii) a link to a network location where the merchandise item of that record is purchasable by a user;

a user-interface module that is configured to enable the user to specify a search criteria that includes or specifies image data;

a search module that performs a search of the collection using the search criteria in order to identify one or more records that satisfy the criteria; and

wherein at least an image of or representing the merchandise item of the record that satisfied the criteria is displayed to the user.

1. A method for providing merchandise items at a network site, the method comprising:

storing a plurality of records, wherein each record includes at least one of (i) data representing an image of a merchandise item, or (ii) the image of the merchandise item, wherein each record further identifies a link to another network site where the merchandise item is purchasable;

enabling a user to enter a search criteria that is based on or includes image data;

performing a search of the plurality of records using the search criteria in order to identify a search result comprising one or more records from the plurality of records; and

displaying at least a portion of the search result to the user.

2. The method of claim 1, wherein enabling a user to enter a search criteria includes enabling the user to specify an image of an object that is to form only a part of a given merchandise item.

3. The method of claim 1, wherein enabling a user to enter a search criteria includes enabling the user to provide a non-textual input that corresponds to a visual characteristic of a given merchandise item or of a portion thereof.

4. The method of claim 3, wherein enabling the user to provide the non-textual input includes:

enabling the user to specify one or more visual characteristics using a non-textual input feature; and

determining an equivalent textual terminology to the one or more visual characteristics.

5. The method of claim 4, wherein enabling the user to specify one or more visual characteristics using the non-textual input feature includes enabling the user to specify a color characteristic using the non-textual input feature.

6. The method of claim 5, wherein enabling the user to specify a color characteristic using the non-textual input feature includes enabling the user to specify the color characteristic using one of a color picker or slider.

7. The method of claim 4, wherein enabling the user to specify one or more visual characteristics using the non-textual input feature includes enabling the user to specify a shading characteristic or a pattern characteristic using the non-textual input feature.

8. The method of claim 7, wherein enabling the user to specify a shading characteristic or a pattern characteristic using the non-textual input feature includes enabling the user to specify one of the shading characteristic, pattern characteristic or combination thereof using a selector or a slider for one or more of a shading and/or a pattern.

9. The method of claim 1, wherein enabling a user to enter a search criteria that is based on or includes image data includes enabling the user to specify an image, or a portion thereof, as the image data.

10. The method of claim 9, further comprising processing the image to determine or identify data representing an object of the image, and using the data representing the object as at least a part of the search criteria.

11. The method of claim 10, wherein performing a search of the plurality of records using the search criteria includes performing a similarity comparison of data representing objects in the plurality of records as compared against the data representing the determined or identified object in the search criteria.

12. The method of claim 11, wherein each of the data representing objects in the plurality of records and data representing the determined or identified object correspond to one or more signature vectors, and wherein performing the similarity comparison includes determining a distance between a signature vector of each of the objects represented by data in the plurality of records and the object represented by the data in the search criteria.

13. The method of claim 9, wherein enabling the user to specify an image, or a portion thereof, as the image data includes enabling the user to specify content provided on a web page.

14. The method of claim 9, wherein enabling the user to specify an image, or a portion thereof, as the image data includes enabling the user to submit an image copied from an external source.

15. The method of claim 9, wherein enabling the user to specify an image, or a portion thereof, as the image data includes enabling the user to submit an image captured by the user.

16. The method of claim 1, wherein enabling a user to enter a search criteria that is based on or includes image data includes enabling the user to specify an image, or a portion thereof, as the image data, and wherein the method further comprises determining a merchandise item, or a class of merchandise items, from the specified image.

17. The method of claim 16, wherein performing a search of the plurality of records using the search criteria includes performing a similarity comparison of data representing objects in the plurality of records as compared against data that represents at least a portion of either the merchandise item or the class of merchandise items.

18. The method of claim 17, wherein performing the similarity comparison includes comparing the data representing objects in the plurality of records to data that represents only a portion or a feature of either the merchandise item or the class of merchandise items.

19. The method of claim 18, wherein enabling the user to specify an image, or a portion thereof, as the image data includes enabling the user to specify content provided on a web page.

20. The method of claim 1, wherein enabling a user to enter a search criteria that is based on or includes image data includes enabling the user to specify at least one of a geometric shape, feature, or a color of the merchandise item.

21. The method of claim 1, wherein enabling a user to enter a search criteria that is based on or includes image data includes enabling the user to specify a visual characteristic of the merchandise item using one of a slider or picker for a feature that corresponds to the visual characteristic.

22. The method of claim 1, further comprising enabling the user to enter text input as an additional search criteria.

23. The method of claim 1, further comprising procuring an image of each of the merchandise items included or represented by data in each of the plurality of records, and analyzing the image of each merchandise item to determine a signature for the merchandise item.

24. The method of claim 23, wherein the data representing the image of the merchandise item corresponds to at least the signature for the merchandise item.

25. The method of claim 1, further comprising analyzing a plurality of content items that each include at least an image of a merchandise item that corresponds at least one of the plurality of records, wherein analyzing the plurality of content items includes generating data representing, for the merchandise item shown in each image, at least one of (i) the image of that merchandise item, (ii) a portion of that merchandise item, or (iii) a visual characteristic of that merchandise item or the portion thereof;

32. The system of claim 31, wherein the user-interface module is configured to enable the user to select or specify an image of an object that comprises at least a portion of the image of the merchandise item, and one or more of (i) a color of the object, or (ii) a shape of the object (iii) pattern of object.

33. The system of claim 32, wherein the user-interface module is configured to enable the user to specify one or more of (i) a manufacturer of the merchandise item, (ii) a price range of the merchandise item, (iii) a keyword for the merchandise item, (iv) a shipping information for the merchandise item, or (v) a pricing information for the merchandise item.

35. The system of claim 31, wherein the user-interface module includes one or more sliders to enable the user to select one or more of a color feature, pattern feature or a shape feature.

36. The system of claim 31, wherein the user-interface module is configured to enable the user to select a portion of a given image as the search criteria.

37. The system of claim 31, wherein the user-interface module includes a color picker that enables the user to select a color of a desired merchandise item.

38. The system of claim 31, wherein at least some of the records in the collection include pricing information, and wherein the search module uses pricing information provided or indicated by the user as part of the search criteria.

39. The system of claim 31, wherein the user-interface module (i) includes one or more user-interface module features for enabling the user to specify input corresponding to one or more of a color, a shape, and a pattern, and (ii) enables the user to select at least one of an image of a merchandise item of interest or a portion thereof, and wherein the search module is configured to combine criteria corresponding to selected input and the image of the merchandise of interest.

40. The system of claim 31, wherein the system is provided to the user over the Internet.

41. The system of claim 31, further comprising an image analysis module that analyzes individual images of merchandise items in order to generate data that represents the individual images of the merchandise items.

42. The system of claim 31, wherein the image analysis module analyzes each image to separate the object or the merchandise item from a background or other objects in the image.

33-1. The system of claim 31, wherein the user-interface module is configured to enable the user to submit or select an image that is not included or represented by data in the collection of records.

43. A method for using image recognition to provide merchandise items on a network site, the method comprising:

obtaining an image of a merchandise item;

programmatically analyzing the image to determine information about the merchandise item; and

using the information to generate a presentation that includes the merchandise item.

44. The method of claim 43, wherein programmatically analyzing the image to determine information about the merchandise item includes determining a color of the merchandise item.

45. The method of claim 43, wherein programmatically analyzing the image to determine information about the merchandise item includes determining a class of the merchandise item.

46. The method of claim 43, wherein programmatically analyzing the image to determine information about the merchandise item includes identifying a local feature of the merchandise item.

47. The method of claim 43, wherein programmatically analyzing the image to determine information about the merchandise item includes identifying a global feature of the merchandise item.

48. The method of claim 43, wherein programmatically analyzing the image to determine information about the merchandise item includes identifying one or more global features and one or more local features.

49. The method of claim 46, wherein identifying one or more local features includes identifying one or more portions of the merchandise item, and identifying a class-specific feature of one of the one or more portions.

50. The method of claim 49, wherein the one or more identified portions include at least one of a key corner or an edge point of the image of the merchandise item.

51. The method of claim 43, further comprising displaying the image of the merchandise item along with a link to a network location where the merchandise item is made available for sale.

52. The method of claim 43, wherein programmatically analyzing the image to determine information includes determining a signature of the merchandise item.

53. The method of claim 52, further comprising determining a classification of the merchandise item.

54. The method of claim 53, wherein determining the classification includes determining whether the merchandise items is a shoe, an item of clothing, a carrying item, a tie, a watch, or a jewelry piece.

55. The method of claim 43, wherein using the information to generate a presentation that includes the merchandise item includes displaying at least a portion of the merchandise item in response to a request that specifies a criteria that is satisfied by the information.

56. The method of claim 55, further comprising storing the information and the merchandise item in a data store that includes information about a plurality of merchandise items, and wherein using the information to generate a presentation includes performing a search operation on the data store to identify a set of merchandise items that include information that satisfy the criteria.

57. The method of claim 55, further comprising storing the information and the merchandise item in an index that includes data about a plurality of merchandise items and information about each of the merchandise items, and wherein using the information to generate a presentation includes performing a search operation using the index to identify a set of merchandise items that include information that satisfy the criteria.

58. A method for providing an online commerce site, the method comprising:

obtaining an image of each of a plurality of merchandise items;

programmatically analyzing each of the plurality of images in order to determine information about that merchandise item, wherein the information includes an attribute or a characteristic of the individual merchandise items;

storing the information about each merchandise item in a data store;

enabling an interface in which a user is able to specify a search criterion that includes a value of the attribute or the characteristic;

in response to receiving the search criterion, performing a search operation to identify one or more merchandise items that satisfy the search criterion.

59. The method of claim 58, wherein enabling an interface in which a user is able to specify a search criterion includes enabling a user to specify a color of the merchandise item.

60. The method of claim 58, wherein enabling an interface in which a user is able to specify a search criterion includes enabling a user to specify a shape or a feature of the merchandise item.

61. The method of claim 58, wherein programmatically analyzing each of the plurality of images in order to determine information about that merchandise item includes storing the information as a quantitative value.

62. The method of claim 61, wherein the quantitative value corresponds to a recognition signature.

63. The method of claim 62, wherein storing the information about each merchandise item in a data store includes storing text information for individual images that that is determined from programmatically analyzing the individual images.

64. The method of claim 63, wherein programmatically analyzing each of the plurality of images in order to determine information about that merchandise item includes identifying a quantitative value from analyzing individual images in the plurality of images, and converting the quantitative value into the text information for the individual images.

65. A method for providing an online commerce site, the method comprising:

providing a collection of records, wherein each record includes (i) an image of a corresponding merchandise item, (ii) image data associated with the image of the merchandise item, and (iii) one or more metadata and/or text items that are descriptive of the merchandise item; and

enabling the user to search the collection of records by specifying, as a search criteria for a desired item, an input corresponding to any of image data, a visual characteristic, a classification, or a keyword.

66. The method of claim 65, enabling the user to search the collection of records includes enabling the user to specify one or more of a brand, a price range, a merchant, a material, a sale amount, a categorization, a style, or a pattern as the input.

67. The method of claim 65, wherein enabling the user to specify the brand or the price range includes enabling the user to enter text or make a menu selection.

68. The method of claim 65, wherein enabling the user to search the collection of records includes enabling the user to specify an image that has image data known to be associated with it.

69. The method of claim 65, wherein enabling the user to search the collection of records includes enabling the user to specify an image that has no image data associated with it, and wherein the method further comprises performing image analysis on the image to identify image data as the input for the criteria.

70. The method of claim 65, wherein enabling the user to search the collection of records includes enabling the user to specify a feature independently.

71. The method of claim 65, wherein enabling the user to search the collection of records includes enabling the user to specify the feature in combination with one or more of a classification or a keyword.

72. The method of claim 65, wherein enabling the user to search the collection of records includes enabling the user to specify the feature in combination with a selected image, and wherein the method further comprises forming a criteria that combines image data from the selected image with the specified feature.

73. The method of claim 70, wherein enabling the user to specify a feature includes enabling the user to specify the feature using non-textual input.

74. The method of claim 70, wherein enabling the user to specify the feature using non-textual input includes enabling the user to specify the feature using one or more of a slider, selector, or picker.

75. The method of claim 65, enabling the user to search the collection of records includes:

enabling the user to perform a first search to obtain a first search result, including displaying to the user a plurality of panels that each contain a merchandise item that satisfies the search criteria of the input; and

enabling the user to perform a second search to obtain a first search result, including enabling the user to select one of the plurality of panels as input.

76. The method of claim 75, further comprising detecting the user selecting one of the plurality of panels, and then identifying image data associated with the selected panel, and using the identified image data as at least part of the search criteria.

77. The method of claim 76, wherein enabling the user to perform a second search includes enabling the user to select one of the plurality of panels in combination with specifying input for one or more features.

78. The method of claim 77, further comprising detecting the user selecting one of the plurality of panels in combination with specifying input for one or more features, then identifying image data associated with the selected panel, and using the identified image data and data corresponding to the one or more features as the search criteria.

79. The method of claim 78, wherein using the identified image data and data corresponding to the one or more features includes weighting the image data of the selection with the input of the one or more features.

80. The method of claim 77, wherein the one or more features correspond to one or more of color, pattern, or shape or metadata.

81. A system for providing merchandise items through a network, the system comprising:

a user-interface configured to enable a user to specify, as a search criteria for a desired item, an input corresponding to any of (i) image data, (ii) a visual characteristic, (iii) a classification, or (iv) a keyword; and

a search module that is configured to access an index for a collection of records, wherein the search module uses the search criteria to determine a search result from the index, the search result identifying one or more merchandise items that satisfy the search criteria.

82. The system of claim 81, wherein the user-interface is further configured to display the search result in the form of one or more panels, wherein each panel identifies the one or more merchandise items and includes an image of the merchandise item and a link to a network location where the merchandise item is available for purchase.

83. The system of claim 82, wherein the user-interface is configured to enable the user to select one of the panels of the search result as at least a part of a subsequent search criteria.

84. The system of claim 83, wherein the search module is configured to identify image data associated with the selected one of the panels as at least a part of the search criteria in identifying a second search result using the index.