Electro-optical system for correlating and integrating image data from frame-to-frame

• US 4,396,903 A
• Filed: 05/29/1981
• Issued: 08/02/1983
• Est. Priority Date: 05/29/1981
• Status: Expired due to Fees

• Current Assignee: Westinghouse Electric Corporation (Paramount Global (f/k/a ViacomCBS Inc.))
• Original Assignee: Westinghouse Electric Corporation (Paramount Global (f/k/a ViacomCBS Inc.))
• Inventors: Helland, Arden R., Habicht, John C.
• Primary Examiner(s): Boudreau, Leo H.
• Application Number:
  US06/268,544
• Time in Patent Office: 795 Days
• Field of Search: 340/146.3 AC, 340/146.3 MA, 340/146.3 H, 340/146.3 R, 340/146.3 Q, 340/146.3 D, 340/146.3 SG, 364/515, 364/444, 364/456, 364/728, 358/93, 358/96, 358/103, 358/105, 358/107, 358/108, 358/109, 343/5 CE, 343/5 CM, 343/5 DP, 343/5 MM, 244/3.15-3.17
• US Class Current: 382/103
• CPC Class Codes: G06V 10/20 Image preprocessing

Abstract

An electro-optical system which integrates the image pattern information, extracted from a plurality of image frames and identified as belonging to a common candidate object, to form a composite image pattern of the candidate object which may be accurately classified with respect to a reference object image pattern is disclosed. More specifically, the formed composite image pattern of the candidate object is correlated with each image pattern of a set of reference object image patterns to generate a corresponding set of correlation values. Preferably, the largest of the correlation values is selected for a comparison with a threshold correlation value. In the event the generated correlation value exceeds the threshold correlation value, the candidate object represented by the composite image pattern is classified as the referenced object corresponding to the selected correlation value. Another aspect of the electro-optical system permits the threshold correlation value to be adaptively selected in accordance with the number of frames integrated to form the composite image pattern under consideration for classification. In addition, a final classification of the candidate object may include the smoothing of a plurality of classification decisions generated throughout the integration sequence during which a number of composite image patterns are formed.

Claims

1. An electro-optical system for processing image information of a plurality of frames, sequentially generated from an electro-optical sensor and representative of a field of view thereof, in order to extract and classify candidate object image patterns in said image frames in accordance with a set of reference object image patterns and a signature feature set associated with each reference pattern, said system comprising:

means operative to extract image patterns belonging to candidate objects from each of the plurality of sequentially generated frames and to derive a set of signature features associated with each image pattern extracted;

first means for correlating said image patterns extracted from frame-to-frame to identify extracted image patterns belonging to common candidate objects;

means for integrating the correlated extracted image patterns belonging to the identified common candidate objects to form a composite image pattern of each candidate object and a composite signature feature set associated therewith;

second means for correlating a formed composite image signature feature set with the signature feature sets of selected reference object image patterns to generate a corresponding set of correlation values;

means for generating a threshold correlation value; and

means operative to classify a candidate object as a reference object based on a comparison between a selected one of said generated correlation values associated with said candidate object and said threshold correlation value.

2. An electro-optical system in accordance with claim 1 wherein the generating means includes means for selecting a threshold correlation value from a predetermined set of values corresponding to the number of sequentially generated frames used by the integrating means in forming the composite image pattern of the candidate object being classified.

3. An electro-optical system in accordance with claim 1 wherein the integrating means includes means for forming a composite image pattern of the candidate object for each frame of an integration sequence of frames by combining the correlated extracted image patterns belonging to the candidate object from the preceding frames of said integration sequence;

wherein the second correlating means includes means for generating a set of correlation values for each frame in said integration sequence based on the formed composite image pattern corresponding to said each frame; and

wherein the classifying means includes means for generating a classification signal for each frame of said integration sequence for which the corresponding selected correlation value exceeds the threshold correlation value.

4. An electro-optical system in accordance with claim 3 wherein the classifying means further includes means for classifying the candidate object as a reference object based on the number of frames in said integration sequence for which a classification signal is generated.

5. An electro-optical system in accordance with claim 3 wherein the threshold correlation value generating means includes means for selecting a threshold correlation value from a predetermined set of values corresponding to each frame in said integration sequence.

6. An electro-optical system in accordance with claim 3 wherein the classifying means includes means for selecting the largest correlation value of the set of correlation values generated for each frame of said integration sequence as the selected candidate object correlation value for comparison with the threshold correlation value of the corresponding frame.

7. An electro-optical system in accordance with claim 1 wherein the extracting means includes means for extracting from a frame of information at least one cluster of picture elements (pixels) having an intensity gradient with their surrounding pixels and for indexing an area of pixels including each cluster with respect to a reference point of said frame;

means for extracting pixels along selected gradient edges within each indexed area to form line segments; and

means for indexing a line segment with respect to a reference point of its corresponding indexed area and for deriving a contrast gradient vector for each extracted line segment of an indexed area, whereby each indexed area constitutes at least a portion of a candidate object and the index and gradient vector of each line segment of said indexed area belong to the set of signature features of said candidate object for each frame.

8. An electro-optical system in accordance with claim 7 wherein the first correlating means includes means for comparing the indexed areas of one frame with those of another frame to identify indexed areas from frame-to-frame belonging to a common candidate object;

and wherein the integrating means includes means for integrating the line segments of indexed areas identified as belonging to a common candidate object to form an indexed area having a composite line segment image.

9. An electro-optical system in accordance with claim 8 wherein the first correlating means includes means for identifying common indexed areas from frame-to-frame by determining an overlap of at least a portion of the indexed areas mutually registered to a common frame reference point.

10. An electro-optical system in accordance with claim 8 wherein the integrating means includes means for superimposing the identified common indexed areas with respect to a common indexed area reference point to integrate the line segments thereof to form the composite line segment image.

11. An electro-optical system in accordance with claim 8 wherein each reference object image pattern includes a set of line segments each having an index and gradient vector associated therewith;

and wherein the second correlating means includes means for correlating the line segments of the integrated composite image of an indexed area with the line segment sets of selected reference object image patterns in accordance with the index and gradient vector signature features of the individual line segments to generate a correlation value for each reference object image pattern.

12. An electro-optical system in accordance with claim 11 wherein the classifying means includes:

means for selecting a generated correlation value for comparison with the threshold correlation value; and

means for classifying the candidate object as the reference object corresponding to said selected correlation value when said selected correlation value exceeds the threshold correlation value.

13. An electro-optical system in accordance with claim 12 where the classifying means includes means for selecting the largest generated correlation value for comparison with the threshold correlation value.