Combined semantic description and visual attribute search

• US 9,317,534 B2
• Filed: 06/17/2013
• Issued: 04/19/2016
• Est. Priority Date: 08/28/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Brandt, Jonathan
• Primary Examiner(s): PENG, HUAWEN A
• Application Number:
  US13/919,312
• Publication Number:
  US 20130282712A1
• Time in Patent Office: 1,037 Days
• Field of Search: 707/731, 707/765
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/9038 Presentation of query results

Abstract

An image search method includes receiving a first query, the first query providing a first image constraint. A first search of a plurality of images is performed, responsive to the first query, to identify a first set of images satisfying the first constraint. A first search result, which includes the first set of images identified as satisfying the first constraint, is presented. A second query is received, the second query providing a second image constraint with reference to a first image of the first set of images. A second search of the plurality of images is performed, responsive to the second query, to identify a second set of images that satisfy the second constraint. A second search result, which includes the second set of images identified as satisfying the second constraint, is presented.

Claims

1. A method comprising:

executing, using a processor of a machine, a first image search based on a submitted scene category, the first image search identifying a first set of images that each have metadata in which a semantic description matches the submitted scene category;

causing a displaying of the first set of images;

executing a second image search based on an image selected from the first set of images, the image being relevant to the first image search and having a visual attribute, the second image search identifying a second set of images that each have the visual attribute and metadata in which the semantic description matches the submitted scene category; and

causing a displaying of the second set of images, the displaying of the second set of images being ordered by a specified one of the submitted scene category or the visual attribute, the specified one of the submitted scene category or the visual attribute being specified by a user selection.

2. The method of claim 1 further comprising:

determining the visual attribute of the image in response to the image being selected from the first set of images.

3. The method of claim 1, wherein:

the executing of the second image search is in response to the image being selected from the first set of images.

4. The method of claim 1, wherein:

the submitted scene category is a user-submitted scene category received from a user device; and

the image selected from the first set of images is a user-selected image.

5. The method of claim 1, wherein:

the submitted scene category is a scene category constraint that includes a semantic description of the submitted scene category.

6. The method of claim 1, wherein:

the submitted scene category is a classification constraint that specifies a category of images in which the first set of images is categorized.

7. The method of claim 1, wherein:

the submitted scene category is a textual tag received in a first submission from a device of a user.

8. The method of claim 1, wherein:

the visual attribute of the selected image is a visual similarity constraint satisfied by each of the second set of images.

9. The method of claim 1, wherein:

the second image search identifies the second set of images as being visually similar to the image selected from the first set of images.

10. The method of claim 1, wherein:

the first image search identifies the first set of images as being categorized based on the submitted scene category.

11. The method of claim 1 further comprising:

receiving the submitted scene category in a first query from a user; and

receiving a selection of the image in a second query from the user.

12. The method of claim 11, wherein:

the first and second sets of images are caused to be displayed on a device of the user from whom the first and second queries are received.

13. A system comprising:

a first search engine configured to execute a first image search based on a submitted scene category, the first image search identifying a first set of images that each have metadata in which a semantic description matches the submitted scene category;

a processor configured by an interface component to cause a displaying of the first set of images; and

a second search engine configured to execute a second image search based on an image selected from the first set of images, the image being relevant to the first image search and having a visual attribute, the second image search identifying a second set of images that each have the visual attribute and metadata in which the semantic description matches the submitted scene category, the executing of the second image search being performed by a processor of a machine;

wherein

the interface component configures the processor to cause a displaying of the second set of images, the displaying of the second set of images being ordered by a specified one of the submitted scene category or the visual attribute, the specified one of the submitted scene category or the visual attribute being specified by a user selection.

14. The system of claim 13, wherein:

the second search engine is configured to determine the visual attribute of the image in response to the image being selected from the first set of images.

15. The system of claim 13, wherein:

the second search engine is configured to execute the second image search in response to the image being selected from the first set of images.

16. The system of claim 13, wherein:

the submitted scene category is a user-submitted scene category received from a user device; and

the image selected from the first set of images is a user-selected image.

17. The system of claim 13, wherein:

the submitted scene category is a scene category constraint that includes a semantic description of the submitted scene category.

18. The system of claim 13, wherein:

the visual attribute of the selected image is a visual similarity constraint satisfied by each of the second set of images.

19. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

executing a first image search based on a submitted scene category, the first image search identifying a first set of images that each have metadata in which a semantic description matches the submitted scene category;

causing the first set of images to be displayed;

executing a second image search based on an image selected from the first set of images, the image being relevant to the first image search and having a visual attribute, the second image search identifying a second set of images that each have the visual attribute and metadata in which the semantic description matches the submitted scene category;

providing an option to specify one of the submitted scene category or the visual attribute to order the second set of images; and

causing the second set of images to be displayed, the displaying of the second set of images being ordered by the specified one of the submitted scene category or the visual attribute.

20. The non-transitory machine-readable storage medium of claim 19, wherein:

determining the visual attribute of the image in response to the image being selected from the first set of images.