Evaluation of edge direction information

• US 20050036688A1
• Filed: 09/03/2001
• Published: 02/17/2005
• Est. Priority Date: 09/04/2000
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Froeba, Bernhard, Kueblbeck, Christian
• Granted Patent: US 8,320,682 B2
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

This invention concerns recognizing and locating a physically demarcated body part by using only a relatively small amount of computation, but still produces a sufficient degree of recognition accuracy. For this purpose a procedure is proposed for detecting physically demarcated body parts (face, hand, leg) of a person'"'"'s image (5) if a body part (2) is depicted in front of a background (3). Borderlines (5d, 5e) in the image are only evaluated along line directions (5a′, 4a′, 4b′, 5c′) to determine, by comparing with model (30), whether the body part image corresponds to a type of body part given by the model. In addition, line directions (5d′, 5e′) inside a body part image and borderline directions (5a) of a physically demarcated body part are used to locate and store its position.

Claims

1. Procedure for recognizing types of physically demarcated body parts (2) like faces, hands, and legs of a person (1) portrayed in a target image (5) when at least one, not yet recognized, body part (2) appears in the target image in front of a background (3) and its position is indicated by means of positional information obtained and stored appropriately;

characterized by

(a) the target image (5) being converted into at least one intermediate image (FIGS. 2b and 2c) on which edge lines (5a, 5b, and 5c) are accentuated, to be evaluated in a following step in regard to their direction (FIG. 2e, 4);

(b) preparing a model (FIG. 8, 30) based on other edge line information obtained previously. The directions of the edge lines in the model are retained for determining which other model or which other edge line information corresponds to the sought-for type of body part (2);

where only edge line directions of a body part image are evaluated in comparison with the model (30) to determine whether the body part image (2) appearing in front of the background (3) corresponds to the sought-for body part and whether the accompanying positional information should be determined and stored;

2. Procedure according to claim 1 when images of several body parts are present and can be consecutively compared with the sought-for type according to their edge line information.

3. Procedure according to claim 1 when the positional information contains data about the position of the sought-for body part in the target grayscale image (5).

4. Procedure according to claim 1 when both the first edge lines (5d, 5e;

5d′

, 5e′

) inside the one (at least) body part image (2) as well as the edge line (5a, 5a′

) of the physically demarcated body part (2) (at least one) are used.

5. Procedure according to claim 1 when a scenic picture is represented which has various locally unchanging intensities and appears as a static image in digital form being a pixel-based representation containing intensity steps.

6. Procedure according to claim 1 when a scenic image has locally changing intensities and shows a subject (1) moving in front of the background (3) in such a way that the relative position or orientation of at least one depicted body part changes with time.

7. Procedure according to claim 1 when boundary line information is seen as edge information which contains data about the orientation, especially if the three-dimensional edges of subject (1) contains lines shown in the two-dimensional intensity image (5).

8. Procedure according to claim 1 when the accentuation of the borderlines is obtained from an evaluation of the intensity difference (contrast) between two adjacent image areas, especially by using a Sobel operator as a type of high-pass plane filter.

9. Procedure according to claim 1 when the type of body part given by Model (30) is found in front of the background (3) in image (5) and made available for finding a binary result, namely whether at least the one type of body part corresponding to the model is to be found in the image or not (detection).

10. Procedure according to claim 1 when the grayscale image (5) is a color picture or one with colored components, in visible light or in an invisible wavelength range.

11. Procedure according to claim 1 when size information is determined and stored in addition to the positional information.

12. Procedure according to claim 11 when size information is made available by changing the relative sizes of the model and the target image (5) and compare them for every pair of sizes to determine similarity values as well as selecting the best similarity index for determining size information.

13. Procedure according to claim 1 when the intermediate image (FIG. 2e, FIG. 4) has been evaluated in terms of direction compared with the model (30) by constructing several sums of angle oriented similarity values, especially the formation of a sum of angle differences between borderline sections of the model and of the intermediate image (FIG. 2e, FIG. 4) evaluated in regard to direction.

14. Procedure according to claim 13 when a threshold value is made available which is compared to all similarity values distributed over the target image, only concluding that a sought-for body part (2) is present in the target image if at least one index is below the minimum threshold value.

20. Procedure according to claim 13 when only a reduced number of terms are used to obtain several sums of similarity indices for each relative position of model (30) and the intermediate image (FIG. 4) which has been evaluated according to direction, especially below 80% of the available summation terms for every relative position when only those similarity data having the best similarity indices are used.

21. Procedure according to claim 13 or claim 20 when only acute angles are compared;

angles between 90° and

180°

are converted to corresponding values between 90° and

0°

before being compared.

22. Procedure according to claim 21 when model (30) contains only directional angles between 0°

and 90°

for borderlines in the model.

15. Procedure for the detection of physically demarcated body parts like a face, a hand or a leg of a person'"'"'s image (5) when a body part (2) is depicted in front of a background (3);

characterized by

borderlines (5d, 5e) of the image are only evaluated along directions (5a′

, 4a′

, 4b′

, 5c′

) to determine by comparison with model (30) whether the body part image corresponds to a type presented by the model, when

at least borderline directions (5d′ and

5e′

) inside at least one body part image and borderline directions (5a) of at least one physically demarcated body part image are used for locating its position and the corresponding positional information is stored.

16. Preparation procedure for the detection of physically demarcated body parts like a person'"'"'s face, hand, or leg in an image (5) when a body part (2) is depicted in front of a background (3) and directional information of the borderlines of at least one body part image and its background for determining in a later step whether the body part image corresponds to a type given by model (30). In preparing for a detection procedure a working image is constructed by calculation for which

(a) the image (5) is converted into another representation (FIG. 2b) which indicates directional data for the borderlines with a clarity value given for each direction;

(b) the further representation is processed by using a threshold value (FIG. 2c) to repress directional data at positions where the clarity is below the threshold value, resulting in reduced directional information (5a′

, 5b′

) in the working image (FIG. 3, FIG. 4);

to enable the working image to be used as model (30) or as target image (FIG. 4).

17. Procedure according to claim 16 when the remaining directional information stays unchanged in the working image, but the clarity indices are set at a standard value for all remaining directions to neutralize the clarity information.

18. Procedure according to claim 17 when the remaining information (the remaining directions) is evaluated in the next step by using model (30) and the position of the body part concerned is located and the positional information is stored.

19. Procedure according to claim 16 when the plane of the representation which has been processed with threshold values, is now processed with a binary operator to obtain a pattern image (FIG. 2d) which is then combined with the next representation.

23. Procedure for locating at least one physically demarcated body part like a face, a hand, or a leg in an image of a person

(i) when at least one body part (2) in the image (5) is depicted in front of a background (3);

(ii) when borderlines (5d, 5e) are accentuated (FIG. 2b), and at least the directions (5a′

, 4a′

, 4b′

, 5c′

) of some of these emphasized borderlines (FIG. 4) have been evaluated by repeated comparisons with model (30) where each comparison occurs at another relative local placement of model (30) and the emphasized borderlines (FIG. 4) to obtain a similarity index for every location;

Selection of a best similarity index from the values obtained in the comparisons;

this determines and stores the position of the body part image.

24. Procedure according to claim 23 when the part of the accentuated borderlines (FIG. 4) forms an intermediate image and the similarity value of a local correspondence is determined by the summation of the similarity data occurring at the borderline angles, especially

by constructing a sum of angle differences between borderline sections of model (30) and the intermediate image (FIG. 2e, FIG. 4) over at least a significant area of model (30);

or

by finding angle differences between borderline sections of model (30) and the intermediate image (FIG. 2e, FIG. 4) with the corresponding trigonometrical conversions and summation of the resulting similarity data for the similarity index.

25. Procedure according to claim 23 when a threshold value is available and compared with all the similarity indices distributed over the plane of the image (5). The presence of a sought-for body part (2) is only confirmed if a value smaller than the threshold minimum is found at least one position.

26. Procedure according to claim 23 when a similarity value is determined for a local correspondence only when using those angle-oriented similarity data for which both model (30) and the intermediate image (FIG. 2e) a directional value is available at the same position and there is no angle (directional information) in at least one of the two images (FIG. 2e, 30), then this contributes nothing to the similarity index at this position.

27. Procedure according to claim 24 when a smaller similarity value means a better or greater similarity.