Method for efficient target detection from images robust to occlusion

• US 8,436,913 B2
• Filed: 09/01/2010
• Issued: 05/07/2013
• Est. Priority Date: 09/01/2009
• Status: Active Grant

• Current Assignee: Fondazione Bruno Kessler
• Original Assignee: Fondazione Bruno Kessler
• Inventors: Lanz, Oswald, Messelodi, Stefano
• Primary Examiner(s): GILES, NICHOLAS G
• Application Number:
  US12/807,388
• Publication Number:
  US 20110050940A1
• Time in Patent Office: 979 Days
• Field of Search: 382/160, 382/205, 382/228, 348/222.1, 348/207.99
• US Class Current: 348/222.1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters

Abstract

The method for efficient target detection from images robust to occlusion disclosed by the present invention detects the presence and spatial location of a number of objects in images. It consists in (i) an off-line method to compile an intermediate representation of detection probability maps that are then used by (ii) an on-line method to construct a detection probability map suitable for detecting and localizing objects in a set of input images efficiently. The method explicitly handles occlusions among the objects to be detected and localized, and objects whose shape and configuration is provided externally, for example from an object tracker. The method according to the present invention can be applied to a variety of objects and applications by customizing the method'"'"'s input functions, namely the object representation, the geometric object model, its image projection method, and the feature matching function.

Claims

1. A method for detecting targets from images which are captured by one or more cameras, said method comprising the steps of:

a) collecting information about geometric and calibration parameters corresponding to each of the one or more cameras;

b) choosing a range of numerical representations defining a range of target configurations to be recognized from the images captured by the one or more cameras, wherein each instance in said range of numerical representations defines a corresponding target state of a plurality of target states associated with the range of numerical representations;

c) selecting a target model that is associated with the chosen range of numerical representations;

d) implementing an image projection procedure that overlays the target model onto an image to define a projected target model for each of the plurality of target states associated with the chosen range of numerical representations;

e) selecting a predetermined range of image features for each of the one or more cameras;

f) implementing a likelihood function between the projected target model and the predetermined range of image features;

g) selecting a first set of a predetermined number of the target states associated with the chosen range of numerical representations to be detected;

h) generating, for a first of the one or more cameras, a feature support map for each image feature in the predetermined range of image features;

i) projecting a first target state of the predetermined number of target states using the image projection procedure and associated calibration information from the first of the one or more cameras to form an image projection;

j) determining from the predetermined range of image features, a range of activated image features associated with a first activated pixel from the image projection procedure;

k) determining a first value associated with how the image feature is projected onto the image;

l) storing said determined first value paired with the projected target state of step (i) in each of the feature support maps associated with said range of activated image features;

m) repeating steps (j) through (l) for a next activated pixel that is activated by the image projection procedure;

n) repeating steps (i) through (m) for a next target state of the predetermined number of the target states to complete the feature support map of each image feature in the predetermined range of image features;

o) repeating steps (i) through (n) for a next camera of the one or more cameras; and

p) processing a first plurality of images captured by the one or more cameras to determine a probabilistic occupancy map from the feature support maps.

2. The method of claim 1, wherein said range of numerical representations for a rigid and vertically symmetric target moving on a horizontal plane is a two dimensional vector defining coordinates of the rigid and vertically symmetric target on the horizontal plane.

3. The method of claim 1, wherein said implementing a likelihood function at step (f) includes a combination of elementary functions that operate on single feature pixels.

4. The method of claim 3, wherein said likelihood function is a sum or product of said combination of elementary functions.

5. The method of claim 1, wherein said step (g) is performed by superimposing a matrix of grid cells on a state space, which is defined by said predetermined range of numerical representations, and choosing central portions of the grid cells as said predetermined number of the target states for a rigid horizontal plane and vertically symmetric target.

6. The method of claim 1, further comprising displaying gray level images for PDL (People Detection and Localization) tasks having a regular state space grid.

7. The method of claim 1, wherein the step of generating the probabilistic occupancy map includes the steps of:

q) applying a feature extraction step on a first image of the plurality of images to identify a set of extracted image features in the predetermined range of image features;

r) receiving an external list of values paired with states for each of one or more external targets;

s) identifying the feature support map associated with a first of the set of extracted image features and the capturing camera;

t) determining a probability value associated with a first element of said identified feature support map and said received external list of values paired with states for each of of the one or more external targets;

u) storing said probability value paired with the state of said first element in the probabilistic occupancy map;

v) repeating steps (t) through (u) for a next element of the identified feature support map;

w) repeating steps (s) through (v) for a next of the set of extracted image features; and

x) repeating steps (q) through (w) for a next of the plurality of images to complete the probabilistic occupancy map.

8. The method of claim 7, wherein said feature extraction of step (q) comprises providing at least one of motion detection and edge detection.

9. The method of claim 7, wherein the external list of values paired with states of step (r) is delivered by an object tracker, wherein for each external target the image projection procedure is implemented.

10. The method of claim 1, wherein the step of determining a first value associated with how the image feature is projected onto the image comprises the steps of:

computing a total quantity of image features in said range of activated image features;

computing a value of an elementary function associated with said first activated pixel for said range of activated image features; and

dividing said elementary function value by a total value associated of said number of activated pixels and a total value associated with said number of activated image features.

11. The method of claim 7, wherein determining said probability value of step t) comprises the steps of:

y1) determining, for each of the one or more external targets, one or more elements of said external list that are closer to the camera which captured said first of a plurality of images relative to said first element of said identified feature support map, and whose image projection overlays said first of the set of extracted image features;

y2) determining an occlusion value from said determined one or more elements of each of the one or more external targets;

y3) multiplying said occlusion value with the first value of said first element of the identified feature support map to define said probability value.

12. The method of claim 11, wherein the step of determining said occlusion value for each of the one or more external targets comprises the steps of:

z1) computing, for a first of the one or more external targets, a sum of values of said determined elements to define a first cumulative value;

z2) repeating step (z1) for a next of the one or more external targets to define a next cumulative value;

z3) computing a product of said cumulative values to define said occlusion value.

13. The method of claim 1, further comprising the step of processing a next plurality of images captured by the one or more cameras to determine a next probabilistic occupancy map from the feature support maps.

14. The method of claim 7, wherein the external list of values paired with states of step (r) is provided manually.

15. The method of claim 1, wherein the step of generating the probabilistic occupancy map includes the steps of:

q) applying a feature extraction step on a first image of the plurality of images to identify a set of extracted image features in the predetermined range of image features;

r) identifying the feature support map associated with a first of the set of extracted image features and the capturing camera;

s) determining a probability value associated with a first element of said identified feature support map;

t) storing said probability value paired with the state of said first element in the probabilistic occupancy map;

u) repeating steps (s) through (t) for a next element of the identified feature support map;

v) repeating steps (r) through (u) for a next of the set of extracted image features; and

w) repeating steps (q) through (v) for a next of the plurality of images to complete the probabilistic occupancy map.

16. The method of claim 15, wherein said feature extraction of step (q) comprises providing at least one of motion detection and edge detection.

17. The method of claim 15, wherein determining said probability value of step (s) comprises the step of setting said probability value to equal the first value of said first element of the identified feature support map.

18. A computer program product for detecting targets from images which are captured by one or more cameras, the computer program product comprising a non-transitory computer readable medium having computer readable program code embodied therein that, when executed by a processor, causes the processor to:

a) collect information about geometric and calibration parameters corresponding to each of the one or more cameras;

b) choose a range of numerical representations defining a range of target configurations to be recognized from the images captured by the one or more cameras, wherein each instance in said range of numerical representations defines a corresponding target state of a plurality of target states associated with the range of numerical representations;

c) select a target model that is associated with the chosen range of numerical representations;

d) implement an image projection procedure that overlays the target model onto an image to define a projected target model for each of the plurality of target states associated with the chosen range of numerical representations;

e) select a predetermined range of image features for each of the one or more cameras;

f) implement a likelihood function between the projected target model and the predetermined range of image features;

g) select a first set of a predetermined number of the target states associated with the chosen range of numerical representations to be detected;

h) generate, for a first of the one or more cameras, a feature support map for each image feature in the predetermined range of image features;

i) project a first target state of the predetermined number of target states using the image projection procedure and associated calibration information from the first of the one or more cameras to form an image projection;

j) determine from the predetermined range of image features, a range of activated image features associated with a first activated pixel from the image projection procedure;

k) determine a first value associated with how the image feature is projected onto the image;

l) store said determined first value paired with the projected target state of step (i) in each of the feature support maps associated with said range of activated image features;

m) repeat steps (j) through (l) for a next activated pixel that is activated by the image projection procedure;

n) repeat steps (i) through (m) for a next target state of the predetermined number of the target states to complete the feature support map of each image feature in the predetermined range of image features;

o) repeat steps (i) through (n) for a next camera of the one or more cameras; and

p) process a first plurality of images captured by the one or more cameras to determine a probabilistic occupancy map from the feature support maps.

19. The computer program product of claim 18, wherein the step of generating the probabilistic occupancy map causes the processor to:

q) apply a feature extraction step on a first image of the plurality of images to identify a set of extracted image features in the predetermined range of image features;

r) receive an external list of values paired with states for each of one or more external targets;

s) identify the feature support map associated with a first of the set of extracted image features and the capturing camera;

t) determine a probability value associated with a first element of said identified feature support map and said received external list of values paired with states for each of of the one or more external targets;

u) store said probability value paired with the state of said first element in the probabilistic occupancy map;

v) repeat steps (t) through (u) for a next element of the identified feature support map;

w) repeat steps (s) through (v) for a next of the set of extracted image features; and

x) repeat steps (q) through (w) for a next of the plurality of images to complete the probabilistic occupancy map.

20. The computer program product of claim 18, wherein the step of generating the probabilistic occupancy map causes the processor to:

q) apply a feature extraction step on a first image of the plurality of images to identify a set of extracted image features in the predetermined range of image features;

r) identify the feature support map associated with a first of the set of extracted image features and the capturing camera;

s) determine a probability value associated with a first element of said identified feature support map;

t) store said probability value paired with the state of said first element in the probabilistic occupancy map;

u) repeat steps (s) through (t) for a next element of the identified feature support map;

v) repeat steps (r) through (u) for a next of the set of extracted image features; and

w) repeat steps (q) through (v) for a next of the plurality of images to complete the probabilistic occupancy map.