Vehicle obstacle detection system

• US 8,611,598 B2
• Filed: 07/26/2011
• Issued: 12/17/2013
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Harman International Shanghai Management Co., Ltd. (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International (China) Holdings Co. Ltd. (Samsung Electronics Co. Ltd.)
• Inventors: Yankun, Zhang, Chuyang, Hong, Weyrich, Norman
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US13/191,308
• Publication Number:
  US 20130027196A1
• Time in Patent Office: 875 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A computer-implemented method of detecting objects in a path of a vehicle is provided. An image frame that depicts the path of the vehicle is obtained. An edge-image corresponding to the image frame is generated. A binary image corresponding to the edge-image is also generated. One or more blobs in the binary image that respectively correspond to one or more objects in the image frame are identified. Based on an analysis of the blobs in the binary image, a determination is made that one of the objects in the image frame is an obstacle in the path of the vehicle.

Claims

1. A computer-implemented method of detecting objects in a path of a vehicle comprising:

obtaining an image frame that depicts the path of the vehicle;

generating an edge-image corresponding to the image frame;

generating a binary image corresponding to the edge-image;

identifying one or more blobs containing one or more components in the binary image that respectively correspond to one or more objects in the image frame;

determining a contour for a component from among the one or more components such that a size of the contour corresponds to a total pixel area of the component; and

determining that an object, corresponding to the component in the image frame, is an obstacle in the path of the vehicle based on at least an edge of the determined contour projecting towards a focus of the image frame.

2. The computer-implemented method of claim 1 further comprising:

generating a warning signal in response to the determination that the obstacle is in the path of the vehicle such that receipt of the warning signal at an output device causes the output device to output an alarm.

3. The computer-implemented method of claim 1 where the image frame is a color image frame and the method further comprising:

generating a grayscale image frame corresponding to the color image frame.

4. The computer-implemented method of claim 1 further comprising:

performing a multi-scale integral image-based segmentation of the image frame to generate the edge-image.

5. The computer-implemented method of claim 4 where the edge-image comprises a grid of pixels, individual pixels having a pixel value and the method further comprising:

comparing the pixel value of the individual pixels in the edge-image to a predetermined pixel value threshold to generate the binary image.

6. The computer-implemented method of claim 5 further comprising:

performing an erosion morphological operation on the binary image to shrink at least one of the one or more blobs in the binary image; and

performing a dilation morphological operation on the binary image to expand at least one of the one or more blobs in the binary image.

7. The computer-implemented method of claim 1 where the binary image comprises a grid of pixels, the method further comprising:

comparing the total pixel area of each component in the binary image to a predetermined pixel area threshold; and

in response to a determination that one or more components have a corresponding total pixel area smaller than the predetermined pixel area threshold, removing the respective blobs of the one or more components from the binary image.

8. The computer-implemented method of claim 1, where the contour is formed as an ellipse, and an axis of the ellipse corresponds to the edge, the computer-implemented method further comprising:

generating a top-view image corresponding to the image frame;

determining prolongation of the axis of the ellipse towards the focus in the top-view image;

determining that the prolongation passes through a predetermined area around the focus; and

in response to the prolongation passing through the predetermined area, determining that the object in the image frame corresponding to the component for the determined ellipse is an obstacle in the path of the vehicle.

9. The computer-implemented method of claim 1 further comprising:

adjusting the image frame to correct for lens distortion in the image frame.

10. A system for detecting objects in a path of a vehicle comprising:

an image segmentation module configured to generate an edge-image corresponding to an image frame, the image frame depicts the path of the vehicle;

a binary image creation module configured to generate a binary image corresponding to the edge-image;

a blob analysis module configured to identify a component corresponding to one or more blobs in the binary image, the component further corresponds to an object in the image frame;

the blob analysis module further configured to determine an ellipse enclosing the identified component; and

an obstacle verification module configured to classify the object corresponding to the identified component as in the image frame an obstacle in the path of the vehicle based on a prolongation of the ellipse passing through a predetermined area around a focus of the image frame.

11. The system of claim 10 where the system is configured to generate a warning signal in response to the determination that the obstacle is in the path of the vehicle such that receipt of the warning signal at an output device causes the output device to output an alarm.

12. The system of claim 10 where the image frame is a color image frame and the system further comprising:

a grayscale transformation module configured to generate a grayscale image frame corresponding to the color image frame.

13. The system of claim 10 where the image segmentation module is configured to perform a multi-scale integral image-based segmentation of the image frame to generate the edge-image.

14. The system of claim 13 where:

the edge-image comprises a grid of pixels, individual pixels having a pixel value; and

the binary image creation module is configured to compare the pixel value of the individual pixels in the edge-image to a predetermined pixel value threshold to generate the binary image.

15. The system of claim 14 where:

the blob analysis module is further configured to perform an erosion morphological operation on the binary image to shrink at least one of the one or more blobs in the binary image; and

the blob analysis module is further configured to perform a dilation morphological operation on the binary image to expand at least one of the one or more blobs in the binary image.

16. The system of claim 10 where a size of the ellipse corresponds to a total pixel area of the component, and where:

the binary image comprises a grid of pixels;

the blob analysis module configured to compare the total pixel area of the component in the binary image to a predetermined pixel area threshold; and

in response to the total pixel area being smaller than the predetermined pixel area threshold, the blob analysis module removes the one or more blobs corresponding to the component from the binary image.

17. The system of claim 10 where:

the obstacle verification module is configured to generate a top-view image corresponding to the image frame;

the obstacle verification module is configured to determine prolongation of an axis of the ellipse towards the focus in the top-view image; and

the obstacle verification module further configured to determine that the prolongation passes through the predetermined area around the focus.

18. The system of claim 10 further comprising a distortion correction module that adjusts the image frame to correct for lens distortion in the image frame.

19. A non-transitory computer readable storage medium comprising instructions executed by a processor, the computer readable storage medium comprising:

instructions to receive an image frame of a path of a vehicle;

instructions to identify a component in a binary image corresponding to one or more blobs detected in the binary image, where the binary image corresponds to the image frame, and the component corresponds to an object in the image frame;

instructions to determine a focus of the image frame;

instructions to classify the object as an obstacle in the path of the vehicle in response to at least an elongated edge of the identified component passing through a predetermined area around the focus.

20. The non-transitory computer readable storage medium of claim 19, where the edge of the identified component is a vertical edge of an enclosing contour of the identified component.