System and method for detecting pedestrians

• US 8,050,459 B2
• Filed: 07/25/2008
• Issued: 11/01/2011
• Est. Priority Date: 07/25/2008
• Status: Expired due to Fees

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Wolski, Mark J., Zhang, Wende
• Primary Examiner(s): Ahmed; Samir
• Assistant Examiner(s): FITZPATRICK, ATIBA O
• Application Number:
  US12/179,645
• Publication Number:
  US 20100021010A1
• Time in Patent Office: 1,194 Days
• Field of Search: 382100-107, 340907-912, 340933-940, 348113-120
• US Class Current: 382/104
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A method is provided for detecting a pedestrian exterior to a vehicle by use of a vehicle-mounted image-based monitoring system. An edge-based pedestrian detection analysis is applied to the input image data to determine an edge-based confidence value of a candidate image area being a pedestrian. A motion-based pedestrian detection analysis is applied to the input image data to determine a motion-based confidence value of a candidate image area being a pedestrian. A final confidence value is obtained by combining the edge-based confidence value and motion-based confidence value using weighting factors that are functions of vehicle speed. The final confidence value is compared to a threshold value for determining whether to generate a warning that a pedestrian may be present in the input image data.

Claims

1. A method for detecting a pedestrian exterior of a vehicle in an image-based monitoring system, the vehicle having an image capture device for capturing images exterior of the vehicle, the image-based monitoring system including a processor for processing image data received from the image capture device for detecting the pedestrian exterior of the vehicle, the method comprising the steps of:

receiving input image data from the image capture device;

applying a pedestrian edge-based detection analysis to the input image data for detecting a pedestrian candidate in the input image data, the edge-based detection analysis determining an edge-based confidence value for indicating a confidence level of the pedestrian candidate being a pedestrian;

applying a pedestrian motion-based detection analysis to the input image data for detecting a pedestrian candidate in the input image data, the motion-based detection analysis determining a motion-based confidence value for indicating a confidence level of the pedestrian candidate being a pedestrian;

determining a speed of the vehicle;

applying a first weighting factor as a function of the vehicle speed to the edge-based confidence value and applying a second weighting factor as a function of the vehicle speed to the motion-based confidence value for determining a final confidence value; and

comparing the final confidence value to a threshold value for determining whether to generate a warning that a pedestrian may be present in the input image data.

2. The method of claim 1 further comprising the step of outputting the image to an image display device for identifying a location of the pedestrian in the captured image.

3. The method of claim 1 wherein applying the edge-based detection analysis for determining the edge-based confidence value comprises the steps of:

generating vertical edge map data from the input map data;

filtering the vertical edge map data for generating a vertical object projection plot;

selecting candidate object regions in response to the vertical object projection plot;

applying horizontal filtering to the candidate object regions for redefining the candidate object regions;

comparing the respective redefined candidate object regions to a predetermined sizing threshold; and

determining the edge-based confidence value in response to comparing the respective redefined candidate object regions and the predetermined sizing threshold.

4. The method of claim 3 wherein the step of filtering the vertical edge map data for generating a vertical object projection plot further comprises the steps of:

comparing projection values of the vertical object projection plot to an upper projection threshold and a lower projection threshold;

saturating respective projection values that exceed the upper projection threshold; and

setting respective projection values that are less than the lower projection threshold to a zero value.

5. The method of claim 4 wherein the step of filtering the vertical edge map data for generating a vertical object projection plot further comprises the step of:

applying a smoothing technique to the vertical object projection plot for generating a plurality of processed projections.

6. The method of claim 5 further comprising the steps of:

determining a horizontal spacing between each respective processed projection; and

merging an adjacent set of processed projections if the horizontal spacing is less than a predetermined spacing.

7. The method of claim 6 wherein the predetermined sizing threshold is based on predetermined pedestrian height characteristics.

8. The method of claim 7 wherein the predetermined sizing threshold is further based on predetermined pedestrian height to width ratio.

9. The method of claim 6 wherein the predetermined sizing threshold is based on predetermined pedestrian head and foot locations.

10. The method of claim 1 wherein applying the motion-based detection analysis for determining the motion-based confidence value comprises the steps of:

generating a motion map from the input image data by determining a displacement of an object between a first instance of time and at a second instance of time;

generating vertical edge map data from the motion map by determining vertical edges of objects in the motion map;

filtering the vertical edge map data for generating a vertical object projection plot;

selecting candidate object regions in response to the vertical object projection plot;

applying horizontal filtering to the candidate object regions for redefining the candidate object regions;

comparing the respective redefined candidate object regions to a predetermined sizing threshold; and

determining the motion-based confidence value in response to comparing the respective redefined candidate object regions and the predetermined sizing threshold.

11. The method of claim 10 wherein the step of generating a motion map from the input image data is performed after Gaussian filtering.

12. The method of claim 10 wherein filtering the vertical edge map data for generating a vertical object projection plot further comprises:

comparing projection values of the vertical object projection plot to a an upper projection threshold and a lower projection threshold;

saturating respective projection values that exceed the upper projection threshold; and

setting respective projection values that are less than the lower projection threshold to a zero value.

13. The method of claim 12 wherein the step of filtering the vertical edge map data for generating a vertical object projection plot further comprises the steps of:

applying a smoothing technique to the vertical object projection plot for generating a plurality of processed projections.

14. The method of claim 13 wherein the smoothing technique includes adding a fraction of a respective vertical object projection of a previous frame to a respective vertical object projection of a current frame.

15. The method of claim 1 wherein final confidence value for comparison with the threshold value is determined by the formula:

f=w_E(v)*f_E+w_M(v)f_M

where f_E is the edge-based confidence value, w_E(V) is a weighting factor based on vehicle speed for the edge-based confidence value, f_M is the motion-based confidence value, w_M(v) is a weighting factor based on vehicle speed for the motion-based confidence value.

16. A pedestrian detection system for detecting a pedestrian exterior of a vehicle in an image-based monitoring system, the system comprising:

a image capture device for capturing images exterior of the vehicle;

a processor for processing input image data received from the image capture device, the processor applying an edge-based detection analysis for determining an edge-based confidence value for indicating a confidence level of the pedestrian candidate being a pedestrian, the processor further applying a motion-based detection analysis to the input image data for determining a motion-based confidence value for indicating a confidence level of the pedestrian candidate being a pedestrian, the processor applying a first weighting factor as a function of a vehicle speed to the edge-based confidence value and applying a second weighting factor as a function of the vehicle speed to the motion-based confidence value for determining a final confidence value, wherein the processor compares the final confidence value to a threshold value for determining whether to generate a warning that a pedestrian may be present in the input image data; and

an image display for displaying an input image data and for identifying a location of the pedestrian on the image display.