AUTOMATIC RECOGNITION OF IMAGES

• US 20120134576A1
• Filed: 01/10/2011
• Published: 05/31/2012
• Est. Priority Date: 11/26/2010
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Banerjee, Serene, Kuchibhotla, Anjaneyulu Seetha Rama, SHARMA, Avinash
• Granted Patent: US 8,744,196 B2
• Time in Patent Office: Days
• US Class Current: 382/155
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/2136 based on sparsity criteria, e.g. with an overcomplete basis
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods

Abstract

Presented is a method of automatically performing an action, based on graphical input. The method comprises: receiving, for a user, an input image; comparing the input image with the contents of a user-customized database comprising a plurality of records, each record representing a predefined class of image, wherein the user has previously associated records in the database with respective specified actions; attempting to recognize the image, based on the similarity of the input image to one of the predefined classes of image represented in the user-customised database; and if the image is recognized, performing the action previously associated by the user with the class. Also presented is apparatus for recognizing an image and a method of constructing a user-customized database.

Claims

1. A method of automatically performing an action, based on graphical input, the method comprising:

receiving, for a user, an input image;

comparing the input image with the contents of a user-customized database comprising a plurality of records, each record representing a predefined class of image, wherein the user has previously associated records in the database with respective specified actions;

attempting to recognize the image, based on the similarity of the input image to one of the predefined classes of image represented in the user-customised database; and

if the image is recognized, performing the action previously associated by the user with the class.

2. The method of claim 1, further comprising:

if the image is not recognized, prompting the user for an action to be associated with a class to which the image belongs; and

adding a record for the class to the database, associated with the action.

3. The method of claim 1, further comprising:

if the image is not recognized, prompting the user to indicate a record in the database to which the image corresponds; and

updating the database, based on the input image, so that the image is in future associated with the same action as the record indicated by the user.

4. The method of claim 1, wherein the step of comparing the input image with the contents of the user-customized database comprises identifying in the image fiducial points suitable for object recognition, comprising:

detecting a first set of points comprising corner points in the image;

detecting a second set of points comprising the contours of edges in the image;

determining the fiducial points as a subset of the points in the first set which are located near to a point in the second set.

5. The method of claim 4, wherein the step of detecting edge contours comprises:

detecting the contours as a set of candidate edge-pixels; and

refining the contours, by eliminating some of the candidate edge-pixels to reduce the width of the contours.

6. The method of claim 4, wherein the step of detecting corner points comprises detecting points which belong to two edges, wherein the angle between the two edges falls in a predefined range about 90 degrees.

7. The method of claim 1, wherein the steps of comparing the input image with the contents of the database and attempting to recognize the image comprise:

classifying the input image into one of a plurality of classes, each class corresponding to one of the records in the database, the classification process comprising;

identifying fiducial points in the test image;

populating a spatial histogram with the fiducial points, to generate a test feature vector;

receiving, for each class, one or more reference feature vectors, extracted from respective one or more reference images for that class;

solving a system of linear equations whereby the test feature vector is modeled as a linear combination of the reference feature vectors; and

assigning the input image to a first class according to the solution.

8. The method of claim 7, wherein:

the step of solving a linear system of equations comprises, for each class, approximating the test feature vector by a linear combination of the one or more reference feature vectors in that class; and

in the step of assigning the test image to a first class, the test image is assigned to the class yielding the most accurate approximation.

9. The method of claim 7, wherein, in the step of populating the spatial histogram, the origin of the histogram is positioned at the centroid of the fiducial points.

10. The method of claim 7, further comprising calculating a measure of confidence that the test image belongs to the assigned first class,

wherein, if the measure of confidence is below a preset threshold, the assignment of the test image to the first class is re-evaluated.

11. The method of claim 10, wherein the re-evaluation of the assignment comprises:

for each of the fiducial points identified in the test image;

selecting that fiducial point; and

populating a spatial histogram with the remaining fiducial points, to generate a test feature vector,

wherein the origin of the histogram is positioned at the selected fiducial point;

receiving, for at least the first class, a reference feature vector for each fiducial point of each reference image in that class;

comparing each of the test feature vectors with each of the reference feature vectors; and

assigning the test image to a class according to the result of the comparison.

13. A computer program comprising computer program code means adapted to control a physical computing device to perform all of the steps of claim 1 if said program is run on a computer.

14. A computer program as claimed in claim 13 embodied on a computer readable medium.

12. A method of constructing a user-customized database, comprising:

receiving, from a user, an input image;

receiving, from the user, an indication of an action to be associated with the image;

creating a record in the database to represent the image; and

associating the record with the action.

15. Apparatus for recognizing an image, comprising:

an input for receiving, from one among a plurality of users, an input image to be recognized;

a first database comprising a plurality of records, each record representing a class of image and associating the class with an action assigned by a first user;

a second database comprising a plurality of records, each record representing a class of image and associating the class with an action assigned by a second, different user;

a comparison unit adapted to compare the input image with the contents of the first database or the second database, wherein the database is selected according to the identity of the user from whom the input image was received;

a recognition processor, adapted to recognize the input image, based on a similarity between the input image and one of the classes represented in the selected database; and

an output, for providing an indication of the action associated with the recognized class.