Method of and device for moving image contour recognition

• US 5,103,488 A
• Filed: 06/07/1990
• Issued: 04/07/1992
• Est. Priority Date: 06/21/1989
• Status: Expired due to Term

• Current Assignee: CSELT Centro Studi e Laboratori Telecomunicazioni SpA (Telecom Italia S.p.A.)
• Original Assignee: CSELT Centro Studi e Laboratori Telecomunicazioni SpA (Telecom Italia S.p.A.)
• Inventors: Masera, Lorenzo, Lettera, Cataldo, Gemello, Roberto
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Fox, David
• Application Number:
  US07/535,115
• Time in Patent Office: 670 Days
• Field of Search: 382/48, 382/22, 382/14, 358/85, 358/96, 358/105, 358/135, 358/136, 358/183
• US Class Current: 382/199
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 9/002 using neural networks
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/94 Vector quantisation

Abstract

The recognition method is applied to visual telephony image coding. Matrices of digital samples relevant to the individual frames of the video transmission are submitted to a first processing whereby the foreground region containing the figure is identified. The information concerning the elements of such a region is then processed by edge recognition algorithms to detect a group of elements possibly belonging to the contour. The group of elements is analyzed to select a sequence of elements distributed on the average along a line. The sequency of elements is processed by a neural network to build up the continuous contour which is then coded.

Claims

1. A method of recognizing the contour of a moving figure forming part of a sequence of images each of which is converted into a matrix of digital signals corresponding to a frame, the method comprising the steps of:

(a) comparing a current frame and a preceding frame to distinguish changed image regions, thereby forming a foreground containing the figure whose contour is to be recognized, and unchanged image regions forming a background, and building up and storing an inter-frame map comprising foreground image elements;

(b) applying an edge recognition algorithm to the foreground region of the current frame to identify image elements which are candidates for the figure contour, and building up and storing an intra-frame map comprising such candidate elements;

(c) forming a logic sum of the inter-frame and intra-frame maps to build up a single map of elements, and scanning such a single map horizontally, vertically and obliquely in scanning steps, each scanning step identifying an element of a possible linear contour of an average unitary width;

(d) vectorially quantizing the elements of said possible linear contour with a neural network to generate respective contour points; and

(e) generating a continuous contour passing through the points obtained by the quantization.

2. The method defined in claim 1 wherein the comparing in step (a) is carried out according to two different techniques, one effecting a comparison of corresponding lines in said frames according to an algorithm of minimization of an error function and the second an evaluation of the brightness difference of corresponding image elements, each comparison producing a respective inter-frame map, said inter-frame maps being logically added together.

3. The method defined in claim 1 wherein said intra-frame map building step (b) further comprises identifying all foreground elements having a brightness value different from the brightness values present in the background and adding elements thus identical to those detected as a result of the application of the edge detection algorithm.

8. The method defined in claim 1 as a phase in a method of coding image signals organized in frames, where only the moving image contour is coded at each frame, while the background is coded once for all frames of the moving image contour.

4. A method of recognizing the contour of a moving figure forming part of a sequence of images each of which is converted into a matrix of digital signals corresponding to a frame, the method comprising the steps of:

(a) comparing a current frame and a preceding frame to distinguish changed image regions, thereby forming a foreground containing the figure whose contour is to be recognized, and unchanged image regions forming a background, and building up and storing an inter-frame map comprising foreground image elements;

(b) applying an edge recognition algorithm to the foreground region of the current frame to identify image elements which are candidates for the figure contour, and building up and storing an intra-frame map comprising such candidate elements;

(c) forming a logic sum of the inter-frame and intra-frame maps to build up a single map of elements, and scanning such a sngle map horizontally, vertically and obliquely in scanning steps, each scanning step identifying an element of a possible linear contour of an average unitary width;

(d) vectorially quantizing the elements of said possible linear contour with a neural network to generate respective contour points; and

(e) generating a continuous contour passing through the points obtained by the quantization, said neural network forming a self-organizing topological map, and vector quantization and continuous contour generation are obtained as a result of a network self-training step, which is repeated for each image in the sequence and during which the self-organizing topological map is caused to converge through successive approximations towards the image contour starting from an initial configuration determined in an initialization phase.

5. The method defined in claim 4 wherein said images are relevant to successive frames of the television image transmission in visual telephony, and the moving figure is the speaker figure.

6. The method defined in claim 4 wherein for said self-training the network is initialized so that said points are uniformly distributed in projection on a straight line located at a base of the figure whose contour is to be identified, the extreme representative points being chosen so that the projections on the straight line of all points in the linear contour lie within a segment defined by such extreme points.

7. The method defined in claim 6 wherein an initialization by a straight line is performed for a first image of the sequence, while for the following images an initial map configuration consists of the contour obtained for a preceding image.

9. An apparatus for recognizing a moving image contour, comprising:

means for converting an image containing a moving figure into a digital sample matrix in the form of a frame; and

means for presenting to a processor device, at each frame time, both a current frame and a preceding frame, said processor device comprising;

comparing means for comparing successive frames of samples and for providing a two-dimensional bit matrix, in a one-to-one correspondence with image elements, wherein a logic value of each bit indicates whether the corresponding element belongs to a background or a foreground region, the foreground region comprising the moving figure;

detecting means connected to said comparing means for detecting, among samples in each frame, those corresponding to elements which are candidates for the moving figure contour;

selecting means connected to said detecting means for selecting, among the candidate elements, elements substantially placed along a line, so as to build a discontinuous contour having an average width equal to an image element; and

a neural network connected to said selecting means to obtain from the discontinuous contour, by successive approximations, a continuous line representing the figure contour.

10. The apparatus defined in claim 9 wherein said comparing means comprises means for comparing the two frames according to different criteria, and means for combining into a single matrix, respective matrices from the different criteria.

11. The apparatus defined in claim 9 wherein said detecting means comprises means for identifying a first set of image elements by evaluating the contrast within said foreground region during a frame, and means for comparing the brightness of the foreground and background elements and for supplying said selecting means with a second set of image elements, comprising elements having a different brightness from that of the background, said first and second set forming the set of candidate elements.

12. The apparatus defined in claim 9 wherein said selecting means comprise means for combining the set of candidate elements and the elements belonging to the foreground region to form a single set of elements, means for scanning the elements of said single set of elements, and means for choosing as discontinuous contour elements a first element of the single set found during each scanning step.

13. The apparatus defined in claim 9 wherein the means forming the matrices receive the images from a TV camera of a videotelephone system, and said processor device recognizes the contour of the speaker and supplies the identified contour devices coding the image signal.