Symbol Classification with shape features applied to neural network

• US 6,731,788 B1
• Filed: 11/17/1999
• Issued: 05/04/2004
• Est. Priority Date: 01/28/1999
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Dimitrova, Nevenka, Agnihotri, Lalitha
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US09/441,949
• Time in Patent Office: 1,630 Days
• Field of Search: 382/157, 382/170, 382/165, 382/159, 382/215, 382/176, 194/206
• US Class Current: 382/157
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

An image processing device and method for classifying symbols, such as text, in a video stream employs a back propagation neural network (BPNN) whose feature space is derived from size, translation, and rotation invariant shape-dependent features. Various example feature spaces are discussed such as regular and invariant moments and an angle histogram derived from a Delaunay triangulation of a thinned, thresholded, symbol. Such feature spaces provide a good match to BPNN as a classifier because of the poor resolution of characters in video streams.

Claims

1. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least two shape-dependent features;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein;

said image processor is programmed to identify feature points in said image;

and said at least two shape-dependent features include a measure of an incidence of angles appearing in a triangulation of said feature points.

2. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least two shape-dependent features;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein;

said image processor is programmed to identify feature points in said image and to form at least one of a Delaunay triangulation and a Voronoy diagram based on said feature points;

and said at least two shape-dependent features include a histogram representing an incidence of angles appearing in said at least one of a Delaunay triangulation and Voronoy diagram.

3. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least two shape-dependent features;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein;

said at least two shape-dependent features include at least one moment from the set;

4. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least two shape-dependent features;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein;

said at least two shape-dependent features include the set of invariant moments;

5. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least one shape-dependent feature;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein said classifier is a text classifier and said feature space includes an angle histogram and at least one invariant moment.

6. A device for classifying symbols in an image data stream containing symbols, comprising:

an image data storage unit with an input connected to capture data from said image data stream and an output;

an image processor, connected to said image data storage unit output, programmed to detect an image coextensive with a symbol to be classified embedded therein;

said image processor including a back propagation neural network (BPNN) trained on a feature space;

said feature space including at least two shape-dependent features;

said image processor being programmed to derive a feature vector from said image based on said feature space and to apply said feature vector to said BPNN to classify said symbol, wherein;

said image processor is programmed to identify feature points in said image and to form at least one of a Delaunay triangulation and a Voronoy diagram based on said feature points;

said derivation of said feature points includes thinning a binarized version of said image;

and said at least two shape-dependent features include a histogram representing an incidence of angles appearing in said at least one of a Delaunay triangulation and a Voronoy diagram.

7. A device for classifying symbols in an image data stream containing symbols, comprising an image processor programmed to calculate invariant moments and applying them to a neural network, said moments including substantially at least the set:

8. The device as claimed in claim 7, wherein said image processor is further programmed to distinguish, before calculating said moments, a first set of pixels forming said image from a second set of pixels not part of said image by forming a connected component from a binarized version of a superimage containing both said first and second sets.

9. The device as claimed in claim 7, wherein said image processor is further programmed to calculate at least one other shape-dependent feature based on feature points derived from said image.

10. A method for classifying symbols in an image data stream containing symbols, said method comprising the steps:

training a back propagation neural network (BPNN) on a feature space including at least two shape-dependent features;

capturing an image from a video data stream;

detecting an image region coextensive with a symbol to be classified embedded therein;

deriving a feature vector from said image based on said feature space; and

applying said feature vector to said BPNN to classify said symbol,

wherein said method further comprises the step;

identifying feature points in said image,

and wherein said at least two shape-dependent features include a measure of an incidence of angles appearing in a triangulation of said feature points.

11. A method for classifying symbols in an image data stream containing symbols, said method comprising the steps:

training a back propagation neural network (BPNN) on a feature space including at least two shape-dependent features;

capturing an image from a video data stream;

detecting an image region coextensive with a symbol to be classified embedded therein;

deriving a feature vector from said image based on said feature space; and

applying said feature vector to said BPNN to classify said symbol,

wherein said method further comprises the steps;

identifying feature points in said image; and

forming at least one of a Delaunay triangulation and a Voronoy diagram based on said feature points,

and wherein said at least two shape-dependent features include a histogram representing an incidence of angles appearing in said at least one of a Delaunay triangulation and a Voronoy diagram.

12. A method for classifying symbols in an image data stream containing symbols, said method comprising the steps:

training a back propagation neural network (BPNN) on a feature space including at least two shape-dependent features;

capturing an image from a video data stream;

detecting an image region coextensive with a symbol to be classified embedded therein;

deriving a feature vector from said image based on said feature space; and

applying said feature vector to said BPNN to classify said symbol, wherein;

said at least two shape-dependent features include at least one moment from the set;

13. A method for classifying symbols in an image data stream containing symbols, said method comprising the steps:

training a back propagation neural network (BPNN) on a feature space including at least two shape-dependent features;

capturing an image from a video data stream;

detecting an image region coextensive with a symbol to be classified embedded therein;

deriving a feature vector from said image based on said feature space; and

applying said feature vector to said BPNN to classify said symbol, wherein;

said at least two shape-dependent features include the set of invariant moments;

14. The method as claimed in claim 13, wherein said symbol is a text character.