SYSTEM AND METHOD FOR OBJECT DETECTION FROM A MOVING PLATFORM

• US 20100202657A1
• Filed: 10/22/2009
• Published: 08/12/2010
• Est. Priority Date: 10/22/2008
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Samarasekera, Supun, Han, Feng, Branzoi, Vlad, Kumar, Rakesh, Salgian, Garbis, Southall, John Benjamin, Jung, Sang-Hack, Xiao, Jiangjian, Eledath, Jayan
• Granted Patent: US 8,744,122 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

The present invention relates to a system and method for detecting one or more targets belonging to a first class (e.g., moving and/or stationary people), from a moving platform in a 3D-rich environment. The framework described here is implemented using a number of monocular or stereo cameras distributed around the vehicle to provide 360 degrees coverage. Furthermore, the framework described here utilizes numerous filters to reduce the number of false positive identifications of the targets.

Claims

1. A system for performing object detection and classification comprising:

one or more detectors configured to identify candidate detections;

one or more filters configured to identify false positives, at least one of which is configured to identify the foreground and background components of the candidate detection; and

one or more classifiers configured to extract classifying features from the identified foreground components and classify the object based on the extracted classifying features.

2. The system of claim 1, wherein the filter that identifies foreground and background components of a candidate region is a contour matching filter.

3. The system of claim 1, wherein the filter that identifies foreground and background components of a candidate region is an infrared filter.

4. The system of claim 1, wherein the classifier is a classifier that uses histogram of oriented gradient feature descriptor.

5. A method of identifying a false positive in a candidate detection, the method comprising the steps of:

extracting binary contours of a potential object within the candidate detection to form an extracted edge map;

matching a local part contour set to the extracted edge map;

refining a set of sampling windows based on the matched local part contour set;

composing a global foreground mask from the extracted edge map;

computing gradient directions for a subset of the area of the candidate detection that correlates with the set of refined sampling windows and the global foreground mask; and

determining, from the gradient directions, whether the candidate detection is a false positive.

6. A method of determining if a candidate detection is a false positive, the method comprising the steps of:

scoring each pixel of the candidate detection based on a thermal intensity derivative of each pixel;

selecting a first threshold score such that a first set of pixels having a low thermal intensity derivative are below the first threshold score, and a second set of pixels having an high thermal derivative score are above the first threshold score;

detecting one or more connected components in the second set of pixels; and

comparing a number of the detected one or more connected components to a total number of pixels in the candidate detection to produce a ratio, wherein if the ratio is above a second threshold score, the candidate detection is identified as a false positive.

7. A system for performing object detection comprising:

a plurality of sensors with a common field of view, wherein frames captured by the plurality of sensors are spatially and temporally aligned to produce a synchronized sequence of frames;

a plurality of detectors configured to identify candidate detections, wherein an active subset of the plurality of detectors is selected based on at least one environmental property, wherein an output of each of the active subset is OR-ed to produce the candidate detections; and

a plurality of filters configured to identify false positives and produce a set of confirmed detections from the candidate detections, wherein an active subset of the plurality of filters is selected based on the at least one environmental property.

8. The system of claim 7, wherein at least one of the active subset of filters is a classifier.

9. The system of claim 8, wherein at least one of the active subset of filters is at least one of an infrared filter, a contour matching filter, a consistency of detection over multiple frames filter, an implausible depth, size, and 3D constraints for detected object filter, a scene classification filter, an activity classification filter, and a Contour and Histogram of Oriented Gradients-based classifier filter.

10. The system of claim 7, wherein the active subset of filters is ordered in parallel, in series, or a combination of both.

11. The system of claim 7, further comprising a visual odometry module for recovering a pose for each frame of the synchronized sequence of frames.

12. The system of claim 7, wherein at least two of the plurality of sensors are of different modalities.

13. The system of claim 7, wherein at least two of the plurality of filters are of different types.

14. The system of claim 7, wherein a modality for a corresponding one of the plurality of sensors is chosen from the list comprising visible, image intensifier, thermal, radar, and time-of-flight sensors.

15. The system of claim 7, wherein the at least one environmental property is at least one of distance to a target, latency requirements, speed of the target, and speed of a mobile platform upon which the system is mounted.

16. The system of claim 7, wherein at least one of the plurality of detectors is at least one of a moving target detector and a slow moving and stationary targets detector.

17. A method for performing object detection, comprising the steps of:

spatially and temporally aligning a plurality of frames captured by a plurality of sensors with a common field of view to produce a synchronized sequence of frames;

identifying candidate detections using an active subset of a plurality of detectors selected based on at least one environmental property;

OR-ing an output of each of the active subset of detectors to produce the candidate detections;

identifying false positives from the candidate detections using an active subset of the plurality of filters selected based on the at least one environmental property; and

combining an output of each of the active subset of the plurality of filters to produce a set of confirmed detections from the candidate detections, wherein an active subset of the plurality of filters is selected based on the at least one environmental property.

18. The method of claim 17, wherein the active subset of the plurality of filters is ordered in parallel, in series, or a combination of both.

19. The method of claim 17, further comprising the step of recovering a pose for each frame of the synchronized sequence of frames using visual odometry.

20. The method of claim 17, wherein the at least one environmental property is at least one of distance to a target, latency requirements, speed of the target, and speed of a mobile platform upon which the plurality of sensors is mounted.

21. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the step of searching a scene for regions of depth that match a template describing the expected size and shape of an object.

22. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the steps of:

extracting binary contours of a potential object within the candidate detection to form an extracted edge map,

matching a local part contour set to the extracted edge map,

refining a set of sampling windows based on the matched local part contour set,

composing a global foreground mask from the extracted edge map,

computing gradient directions for a subset of the area of the candidate detection that correlates with the set of refined sampling windows and the global foreground mask, and

determining, from the gradient directions, whether the candidate detection is a false positive.

23. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the steps of:

scoring each pixel of the candidate detection based on a thermal intensity derivative of each pixel,

selecting a first threshold score such that a first set of pixels having a low thermal intensity derivative are below the first threshold score, and a second set of pixels having an high thermal derivative score are above the first threshold score,

detecting one or more connected components in the second set of pixels, and

comparing a number of the detected one or more connected components to a total number of pixels in the candidate detection to produce a ratio, wherein if the ratio is above a second threshold score, the candidate detection is identified as a false positive.

24. The method of claim 23, further comprising the step of computing the area of pixels and the number of connected components, wherein the candidate detection is identified as a false positive when the number of connected components divided by the area is greater than or equal to a second threshold.

25. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the steps of associating instantaneous detections over time, wherein only those candidate detections that persist for a predetermined number of frames are confirmed as being objects of interest.

26. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the steps of:

determining the range to a candidate detection from at least one of motion stereo and the target image location and ground plane equation;

receiving a candidate detection image size; and

identifying the candidate detection as a false-positive when the image size, range to the candidate detection, and nominal size of the candidate detection are inconsistent with each other.

27. The method of claim 17, wherein the step of identifying false positives from the candidate detections further comprises the step of confirming candidate detections based on the candidate detection performing a specific activity.

28. The method of claim 27, wherein the step of confirming candidate detections based on the candidate detection performing a specific activity further comprises the steps of:

analyzing the spatio-temporal volume for the candidate detection to determine the type of activity it represents, and

selected as the underlying activity the embedded perceptual distance to a number of typical exemplars.