IMAGE SEARCH SYSTEM AND METHOD FOR PERSONALIZED PHOTO APPLICATIONS USING SEMANTIC NETWORKS

• US 20150026101A1
• Filed: 07/17/2013
• Published: 01/22/2015
• Est. Priority Date: 07/17/2013
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Marchesotti, Luca, Lin, Julianna Elizabeth
• Granted Patent: US 9,330,110 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

A system and method for searching a finite collection of images using at least one semantic network. Upon receipt of a query from a user that includes a theme and one or more initial keywords, a set of keywords based on the theme and including the initial keywords is generated from one or more semantic networks corresponding to the theme and/or initial keywords. When the finite collection of images includes suitable metadata, a result set is generated of images corresponding to the expanded set of keywords. When the finite collection includes images lacking in metadata, a remote third-party image collection is searched with the set of keywords to obtain a result set that is used to train visual classifiers as to visual concepts associated with the keywords. The classifiers are used to classify the images in the finite collection lacking metadata and the search of the finite collection is performed with the set of keywords to generate a result set.

Claims

1. A method for searching a finite collection of images using at least one semantic network, comprising:

receiving a query from an associated user, the query comprising a theme;

identifying a set of keywords associated with the theme, each of the set of keywords being linked to the theme via the at least one semantic network;

searching the finite collection of images with the identified set of keywords; and

retrieving a result set of images corresponding to the theme from the finite collection of images responsive to a result of the searching thereof,

wherein at least one of the receiving, identifying, searching, and retrieving is performed by a computer processor.

2. The method according to claim 1, wherein the finite collection of images includes a plurality of images having metadata associated therewith and at least one image without metadata.

3. The method according to claim 2, further comprising:

retrieving a set of images from a remote collection of images corresponding to the identified set of keywords;

training a plurality of visual classifiers to learn a set of visual concepts corresponding to the retrieved set of images;

selecting a set of visual classifiers for application to each image in the finite collection of images without metadata; and

classifying each image in the finite collection of images without metadata in accordance with the selected set of visual classifiers; and

updating the result set of images from the finite collection with at least one classified image.

4. The method according to claim 3, wherein the remote collection of images comprises at least one-third party image repository, and wherein each image in the remote collection of images includes metadata.

5. The method according to claim 4, wherein the query includes at least one initial keyword, and wherein set of keywords includes the at least one initial keyword.

6. The method according to claim 3, wherein selecting the set of visual classifiers further comprises:

testing each of the plurality of visual classifiers with respect to accuracy thereof, wherein selecting the set of visual classifiers includes selecting each visual classifier testing above a predetermined threshold.

7. The method according to claim 6, further comprising:

classifying each image in the finite collection of images in accordance with the selected set of visual classifiers; and

determining a classification score associated with each image in the finite collection of images,

wherein retrieving the set of result images comprises retrieving each image of the finite collection having a classification score above a predetermined value.

8. The method according to claim 7, wherein the at least one semantic network includes a plurality of nodes and at least one link between each of the plurality of nodes, each node corresponding to a keyword.

9. The method according to claim 8, wherein identifying the set of keywords further comprises:

determining a weight associated with each link between nodes associated with the theme; and

identifying the set of keywords corresponding to each node having an associated weight greater than a predetermined value.

10. The method according to claim 7, wherein each image in the finite collection of images includes a weight corresponding to a number of keywords associated therewith, and wherein retrieving the result set of images corresponding to the theme from the finite collection of images responsive to a result of the searching further comprises retrieving images having greater weights.

11. The method according to claim 10, wherein classifying each image in the finite collection of images further comprises adjusting the weight of an image in the finite collection of images responsive to a selection thereof by a user.

12. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer causes the computer to perform the method of claim 1.

13. A system comprising memory storing instructions for performing the method of claim 1, and a processor in communication with the memory which implements the instructions.

14. A system for searching a finite collection of images using at least one semantic network, comprising:

a keyword identifier component configured to identify a set of keywords associated with a theme using the at least one semantic network;

a classification component configured to classify each image in the finite collection of images according to a visual concept;

memory which stores instructions for;

retrieving, from a remote collection of images, at least one image corresponding to each of the set of keywords,

training a plurality of classifiers for the classification component, each of the plurality of classifiers corresponding to a visual concept associated with at least one of the images retrieved from the remote collection,

classifying each image in the finite collection of images in accordance with a selected set of classifiers,

searching the finite collection of classified images in accordance with the identified set of keywords, and

retrieving a result set of images corresponding to the theme and the set of keywords from the finite collection of images responsive to a result of the searching thereof; and

a processor in communication with the memory which executes the instructions.

15. The system according to claim 14, wherein the finite collection of images is stored in a local data storage and the remote collection of images is stored in a remote third-party data storage.

16. The system according to claim 14, wherein the memory stores instructions for:

testing each of the plurality of classifiers with respect to accuracy thereof, wherein the set of classifiers includes each classifier testing above a predetermined threshold.

17. The system according to claim 16, wherein the classification component is further configured to determine a classification score associated with each image in the finite collection of images, and wherein retrieving the set of result images comprises retrieving each image of the finite collection having a classification score above a predetermined value.

18. The system according to claim 17, further comprising a weighting component configured to increase or decrease a weight of an association between a keyword and an image in the finite collection of images, wherein the weight is used by the classification component in determining the classification score associated with the image.

19. The system according to claim 18, wherein the classification component is further configured to generate metadata for at least one image in the finite collection of images in accordance with the classification thereof, wherein the classification includes a weighted association of at least one keyword with the at least one image.

20. A computer-implemented method for searching a finite collection of images using at least one semantic network, comprising:

receiving, by a computer system having a processor, a query from an associated user, the query including a theme and at least one initial keyword;

identifying a set of keywords associated with the received theme and at least one initial keyword using at least one semantic network associated with the received theme and the at least one initial keyword;

retrieving a set of images from a remote data storage having metadata including at least one of the set of semantically linked keywords;

training a plurality of visual classifiers to learn a set of visual concepts corresponding to the retrieved set of images;

classifying each of a plurality of images of the finite collection of images in accordance with the trained visual classifiers;

generating a result set of images from the finite collection of classified images in response to a search thereof with the set of keywords