Detecting pedestrians using patterns of motion and apprearance in videos

• US 20040258307A1
• Filed: 06/17/2003
• Published: 12/23/2004
• Est. Priority Date: 06/17/2003
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Jones, Michael J., Viola, Paul A.
• Granted Patent: US 7,212,651 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A method detects a moving object in a temporal sequence of images. Images are selected from the temporally ordered sequence of images. A set of functions is applied to the selected images to generate a set of combined images. A linear combination of filters is applied to a detection window in the set of combined images to determine motion and appearance features of the detection window. The motion and appearance features are summed to determine a cumulative score, which enables a classification of the detection window as including the moving object.

Claims

1. A method for detecting a moving object in a temporal sequence of images of a video, comprising:

selecting images from the temporally ordered sequence of images;

applying a set of functions to the selected images to generate a set of combined images;

evaluating a linear combination of filters on a detection window in the set of combined images to determine motion and appearance features of the detection window;

summing the motion and appearance features to determine a cumulative score; and

classifying the detection window as including the moving object when the cumulative score is greater than a predetermined threshold

2. The method of claim 1 wherein the moving object is a pedestrian.

3. The method of claim 1 wherein the selected images are an adjacent pair of images.

4. The method of claim 1 wherein the selecting, applying, evaluating, summing and classifying is repeated for the entire video.

5. The method of claim 1 further comprising:

partitioning each combined image into a set of detection windows; and

repeating the evaluating, summing, and classifying for each detection window.

6. The method of claim 1 wherein internal components of a particular filter is rectangular and arranged on a diagonal.

7. The method of claim 6 wherein each rectangle filter is sized to match the associated detection window.

8. The method of claim 1 wherein a particular feature h_j is evaluated to a weight α

_j or β

_j according to

9. The method of claim 8 wherein a particular cumulative sum C(x) is assigned a value 1 or 0 according to

10. The method of claim 1 wherein a particular combined image is a difference of the selected images.

11. The method of claim 1 wherein a particular combined image is a difference of shifted selected images.

12. The method of claim 11 wherein the shifting is linear.

13. The method of claim 11 wherein the shifting is rotational.

14. The method of claim 11 wherein the shifting is in scale.

15. The method of claim 1 wherein the set of functions are defined with respect to a scale of the detection window.

16. The method of claim 1 further comprising:

determining a direction of motion of the moving object.

17. The method of claim 1 further comprising:

determining a magnitude of motion of the moving object.

18. The method of claim 1 wherein the filter is a rectangle filer.

20. The method of claim 1 wherein the filter is non-linear.

19. The method of claim 19 wherein the filter is linear.