SYSTEM AND METHOD FOR SCENE TEXT RECOGNITION

• US 20150063688A1
• Filed: 09/05/2014
• Published: 03/05/2015
• Est. Priority Date: 09/05/2013
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Piramuthu, Robinson, Jagadeesh, Vignesh, Di, Wei, Lee, Chen-Yu
• Granted Patent: US 9,245,191 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

Apparatus and method for performing accurate text recognition of non-simplistic images (e.g., images with clutter backgrounds, lighting variations, font variations, non-standard perspectives, and the like) may employ a machine-learning approach to identify a discriminative feature set selected from among features computed for a plurality of irregularly positioned, sized, and/or shaped (e.g., randomly selected) image sub-regions.

Claims

1. A character-recognition method for discriminating between a plurality of character classes, the method comprising:

using a processor,

defining a plurality of irregularly sized and positioned sub-regions within an image region of specified image dimensions;

for each of the plurality of character classes, determining a discriminative feature space by

computing, for each of a plurality of training images within a set of training images having the specified image dimensions and being associated with the character class, mid-level features for all of the sub-regions;

computing feature weights for the mid-level features using a machine-learning algorithm applied to the set of training images;

ranking the mid-level features based on the weights; and

selecting a number of top-ranked features for inclusion in the discriminative feature space; and

creating a combined feature space from the discriminative feature spaces for all of the character classes.

2. The method of claim 1, further comprising, for each of the plurality of character classes, computing updated feature weights for all mid-level features within the combined feature space.

3. The method of claim 2, further comprising classifying a scene character based on an image thereof by

computing, from the scene character image, mid-level features for the sub-regions within the combined feature space;

computing, for each of the plurality of character classes, a weighted average of the mid-level features using the updated feature weights; and

comparing the weighted averages computed for the plurality of character classes to identify, among the plurality of character classes, a closest match for the scene character.

4. The method of claim 1, wherein the sub-regions are rectangular and wherein defining the plurality of irregularly sized and positioned sub-regions comprises randomly selecting a position, a width, and a height from respective uniform distributions.

5. The method of claim 1, further comprising, for each of the plurality of training images within the set, computing pixel-wise low-level features, wherein computing the mid-level features comprises aggregating the low-level features over the pixels within each of the sub-regions.

6. The method of claim 1, wherein the mid-level features comprise, for each of the sub-regions, features computed for a plurality of feature channels.

7. The method of claim 6, wherein the feature channels comprise three color channels, six gradient histogram channels, and one gradient-magnitude channel.

8. The method of claim 1, wherein the machine-learning algorithm comprises a supervised learning algorithm.

9. The method of claim 8, wherein the supervised learning algorithm comprises a support vector machine algorithm.

10. The method of claim 1, wherein the plurality of character classes comprises 52 English letter classes and ten number-digit classes.

11. A system comprising:

a plurality of modules, comprising one or more processors, comprising;

a sub-region-selection module configured to define a plurality of irregularly sized and positioned sub-regions within an image region;

a feature-computation module configured to compute, from an input image, mid-level features for each of a specified plurality of sub-regions;

a machine-learning module configured to determine, from mid-level features computed for a plurality of input images of a set of training images associated with a character class, feature weights associated with the character class; and

a feature-selection module configured to rank features for a character class based on the feature weights associated with the class, and to select a number of top-ranked features for inclusion in a discriminative feature space.

12. The system of claim 11, wherein the plurality of modules further comprises:

a training-control module configured to process a plurality of training images associated with a plurality of character classes by

causing the feature-computation module to compute, for each of the training images, mid-level features for each of the sub-regions defined by the sub-region-selection module,

causing the machine-learning module to determine, for each of the character classes, feature weights associated therewith,

causing the feature-selection module to select top-ranked features for inclusion in the discriminative feature space for each of the character classes.

13. The system of claim 12, wherein the training-control module is further configured to cause the machine-learning module to re-determine, for each of the character classes, feature weights for all features included in the discriminated feature space.

14. The system of claim 11, wherein the plurality of modules further comprises:

a character-classification module configured to cause the feature-computation module to compute, from an input image containing a scene character, mid-level features for each of a plurality of sub-regions within the discriminative feature space, and to classify the scene character based on the mid-level features and feature weights associated with the character classes.

15. The system of claim 14, further comprising a camera for acquiring images containing scene characters.

16. The system of claim 11, wherein the sub-region-selection module is configured to define rectangular sub-regions, defining the sub-regions comprising randomly selecting positions, widths, and heights thereof from respective uniform distributions.

17. The system of claim 11, wherein the feature-computation module comprises a low-level-feature-computation module configured to compute pixelwise features for one or more feature channels and a feature-pooling module configured to aggregate the pixelwise features across the specified sub-regions.

18. The system of claim 11, wherein the feature-computation module is configured to compute features for a plurality of feature channels, the channels comprising three color channels, six gradient histogram channels, and a gradient magnitude channel.

19. The system of claim 11, wherein the machine-learning module is configured to implement a support vector machine.

20. A non-transitory machine-readable medium comprising a plurality of machine-executable instructions configured to cause one or more processors to:

define a plurality of irregularly sized and positioned sub-regions within an image region of specified image dimensions;

for each of the plurality of character classes, determine a discriminative feature space by

computing, for each of a plurality of training images within a set of training images having the specified image dimensions and being associated with the character class, mid-level features for all of the sub-regions;

computing feature weights for the mid-level features using a machine-learning algorithm applied to the set of training images;

ranking the mid-level features based on the weights; and

and selecting a number of top-ranked features for inclusion in the discriminative feature space; and

create a combined feature space from the discriminative feature spaces for all of the character classes.