COMPUTER-IMPLEMENTED METHOD, A COMPUTER PROGRAM PRODUCT AND A COMPUTER SYSTEM FOR IMAGE PROCESSING

• US 20130170738A1
• Filed: 06/30/2011
• Published: 07/04/2013
• Est. Priority Date: 07/02/2010
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Fabio Mammoliti
• Inventors: Capuozzo, Giuseppe, Borghini, Marco, Mammoliti, Fabio
• Granted Patent: US 9,349,077 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The present description refers in particular to a computer-implemented method, a computer program product and a computer system for image processing, the method comprising: receiving at least one user image; identifying a plurality of image classification elements of the user image by: assigning an initial classification to the user image, wherein the initial classification is based on temporal data associated with the user image; determining at least one image label that globally describes content of the user image; calculating a label correctness value for each image label; recognizing at least one image component of the user image; calculating a component correctness value for each image component; correlating the image label and the image component using the label correctness value and the component correctness value, whereby a correlated image label and a correlated image component are identified; applying a rule to determine a category of the user image, wherein the rule is based on at least one of the following: the temporal data, the correlated image label and the correlated image component; and producing a final classification of the user image including the following image classification elements: the initial classification, the correlated image label, the correlated image component, and the category.

Claims

1. A computer-implemented method for image processing, the method comprising:

receiving (701) at least one user image (506);

identifying a plurality of image classification elements of the user image by;

assigning an initial classification to the user image, wherein the initial classification is based on temporal data associated with the user image;

determining at least one image label that globally describes content of the user image;

calculating a label correctness value for each image label;

recognizing (705) at least one image component (508, 510) of the user image;

calculating a component correctness value for each image component (508, 510);

correlating (708) the image label and the image component (508, 510) using the label correctness value and the component correctness value, whereby a correlated image label and a correlated image component are identified;

applying a rule to determine a category of the user image, wherein the rule is based on at least one of the following;

the temporal data, the correlated image label and the correlated image component; and

producing a final classification of the user image including the following image classification elements;

the initial classification, the correlated image label, the correlated image component, and the category.

2. The method of claim 1, wherein identifying the plurality of image classification elements further comprises:

receiving a geographic location associated with the image; and

determining a place name associated with the geographic location;

wherein the final classification further includes the place name.

3. The method of claim 2, wherein identifying the plurality of image classification elements further comprises:

determining an event based on the temporal data and the geographic location;

wherein the final classification further includes the event.

4. The method of any one of the preceding claims, wherein identifying the plurality of image classification elements further comprises:

deriving a weather indicator from the temporal data;

wherein the final classification further includes the weather indicator.

5. The method of any one of the preceding claims, wherein recognizing at least one image component (508, 510) of the user image (506) further comprises recognizing a plurality of image components, and wherein identifying the plurality of image classification elements further comprises:

associating an image component classification with a first image component of the plurality of image components;

wherein the final classification further includes the image component classification.

6. The method of claim 5, wherein the first image component is recognized as a face, the method further comprising:

associating a name with the face; and

determining a mood based on an expression of the face;

wherein the final classification further includes the name and the mood.

7. The method of any one of the preceding claims, further comprising:

verifying the initial classification;

verifying the final classification of the user image, wherein training information is received for producing a subsequent final classification of a subsequent image.

8. The method of any one of the preceding claims, further comprising:

associating the user image with a stored image based on the initial classification; and

/or

associating the user image with a stored image based on the final classification.

9. The method of any one of the preceding claims, wherein receiving the at least one user image further comprises:

receiving a plurality of user images; and

preferably

retrieving (804) at least one of the plurality of user images from an image sharing network.

10. The method of claim 9, further comprising:

displaying a plurality of image classification elements, wherein each image classification element is displayed according to a number of the plurality of user images associated with the image classification element;

receiving user input selecting a selected image classification element from the plurality of image classification elements;

showing a preview of a selected image from the plurality of user images, wherein the selected image classification element is included in the final classification of the selected image.

11. The method of claim 9 or 10, further comprising:

receiving (901) a query including at least one query term;

matching (905) the query term to a matching classification element;

retrieving (909) a matching image from the plurality of user images, wherein the matching classification element is included in a final classification of the matching image.

12. The method of any one of claims 9 to 11, wherein the plurality of user images comprises a query image and a response image, the method further comprising:

receiving a query comprising the query image;

matching a classification element of the query image with a classification element of the response image; and

retrieving the response image in response to the query.

13. A computer program product comprising computer-readable instructions, which, when loaded and executed on a computer system, causes the system to perform operations according to the method of any one of the preceding claims.

14. A client-server system operable to process images, the system comprising:

a client (202), the client (202) comprising;

a receiving module operable to receive a user image (506);

a client image processing module (212) operable to assign an initial classification to the user image, wherein the initial classification is based on temporal data associated with the user image (506);

a server (204), the server (204) comprising;

a server communication module (222) operable to receive the user image (506) from the client (202);

a scene detector (404) operable to identify an image label that globally describes the user image (506) and further operable to calculate a label correctness value for the image label;

an object identifier (406) operable to recognize an image component (508, 510) of the user image (506) and further operable to calculate a component correctness value for the image component (508, 510);

a correlation module (408) operable to correlate the image label and the image component (508, 510) using the label correctness value and the component correctness value, and further operable to identify a correlated image label and a correlated image component;

a categorization module (412) operable to apply a rule in order to determine a category of the user image (506), wherein the rule is based on at least one of the following;

the temporal data, the correlated image label, and the correlated image component; and

a final classification module operable to produce a final classification based on the initial classification, the correlated image label, the correlated image component and the category;

wherein the server communication module (222) is further operable to send the following to the client;

the final classification and a reference to the user image (506).