Method for identifying vehicles in electronic images

• US 7,676,087 B2
• Filed: 09/20/2006
• Issued: 03/09/2010
• Est. Priority Date: 09/20/2006
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Zhang, Yan, Dhua, Arnab S., Kiselewich, Stephen J.
• Primary Examiner(s): Do; Anh Hong
• Application Number:
  US11/523,986
• Publication Number:
  US 20080069402A1
• Time in Patent Office: 1,266 Days
• Field of Search: 382/104, 382/199, 382/264, 382/170, 382/181, 348/625, 348/607, 348/627, 348/521, 348/92, 348/125, 358/3.2, 347/131, 725/127
• US Class Current: 382/170
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A method for identifying objects in an electronic image is provided. The method includes the steps of providing an electronic source image and processing the electronic source image to identify edge pixels. The method further includes the steps of providing an electronic representation of the edge pixels and processing the electronic representation of the edge pixels to identify valid edge center pixels. The method still further includes the step of proving an electronic representation of the valid edge center pixels. Each valid edge center pixel represents the approximate center of a horizontal edge segment of a target width. The horizontal edge segment is made up of essentially contiguous edge pixels. The method also includes the steps of determining symmetry values of test regions associated with valid edge center pixels, and classifying the test regions based on factors including symmetry.

Claims

1. A method for identifying objects in an electronic image, comprising the steps of:

using a camera for providing an electronic source image generated with an array of pixels;

using a processor for;

electronically processing the electronic source image to identify edge pixels that make up edge segments of the electronic source image;

providing an electronic representation of the edge pixels that make up corresponding edge segments of the electronic source image; and

processing the electronic representation of the edge pixels to identify valid edge center pixels in at least one row and provide an electronic representation of the valid edge center pixels, wherein each valid edge center pixel is the approximate horizontal center of a horizontal edge segment of essentially contiguous edge pixels having a target width.

2. The method of claim 1, further comprising the step of determining symmetry values for test regions in the electronic source image, wherein the location of the test regions in the electronic source image is based on the location of valid edge center pixels in the electronic representation of valid edge center pixels, and wherein each test region is bounded on the bottom by a horizontal edge segment of essentially contiguous edge pixels centered on a valid edge center pixel, and wherein each test region is approximately rectangular, and has a predetermined height measured from the bottom of the test region.

3. The method of claim 2, wherein a symmetry value of a test region is determined by multiplying individual pixels of the test region by corresponding individual pixel values of a mirror image of the test region, adding the products of the multiplication together, and normalizing the sum to provide a symmetry value for the test region.

4. The method of claim 3, wherein a filtered version of the test region and a mirror image of the filtered version of the test region are used to determine a symmetry value for the test region.

5. The method of claim 2, wherein the horizontal edge segment of essentially contiguous edge pixels centered on a valid edge center pixel has a width approximately equal to an expected width of a vehicle image in the row in which the horizontal image edge segment is located, and wherein the predetermined height is an expected height of a vehicle image having the expected width.

6. The method of claim 2, further comprising the step of identifying for at least one segment of essentially contiguous edge pixels a symmetrical valid edge center pixel associated with the test region having the highest symmetry value.

7. The method of claim 6, further comprising the step of classifying test regions associated with symmetrical valid edge center pixels to identify test regions of interest, wherein a test region of interest is a test region having characteristics consistent with an object to be detected.

8. The method of claim 7, wherein the characteristics of a test region of interest include at least one of high symmetry and low entropy.

9. The method of claim 7, wherein test regions are classified on a row-by row basis moving from bottom-to-top in the image, and wherein test regions located above already classified test regions of interest are not classified.

10. The method of claim 7, further comprising the step of programming a neural network, wherein the neural network is used to classify the test regions.

11. The method of claim 10, wherein the factors used by the neural network to classify a test region include at least one of the symmetry of the region, entropy of the pixels in the region, ratio of edge pixels in the region to total edge pixels in the region, and the ratio of edge pixels in the region to the number of pixels in the region.

12. The method of claim 7, wherein test regions are classified as strong regions of interest, weak regions of interest, and regions of non-interest, and wherein classification of test regions is discontinued in areas above strong regions of interest.

13. The method of claim 7, further comprising the step of refining the boundaries of identified test regions of interest to create refined regions of interest having at least one of higher symmetry or lower entropy relative to unrefined identified test regions.

14. The method of claim 13, wherein the boundaries of identified test regions of interest are refined by altering at least one of the width, height, aspect ratio, vertical location and horizontal location of the identified test region of interest.

15. The method of claim 13, further comprising the step of providing an indication of the location and size of the refined regions of interest to additional processing circuitry for processing to identify vehicles.

16. The method of claim 1, further comprising the step of filtering the electronic representation of the source image by rows to remove from the electronic representation of the image at least one of angled image edge segments having an angle greater than a predetermined angle value and horizontal image edge segments having a length substantially greater than a predetermined width.

17. The method of claim 16, wherein the predetermined angle value is 5 degrees, and wherein the predetermined width is approximately 2.5 times the width a vehicle image is expected to have in the row in which the horizontal image edge segment is located.

18. The method of claim 1, wherein the electronic representation of edge pixels comprises a two-dimensional matrix of values indicating which locations in the electronic source image are edge pixels.

19. The method of claim 1 further comprising the step of vertically thickening identified edge pixels associated with edge segments of the electronic source image prior to identifying valid edge center pixels by causing pixels at least one of immediately above and immediately below identified edge pixels to take on the value of an edge pixel in the electronic representation of edge pixels.

20. The method of claim 1, wherein the step of processing the electronic representation of the edge pixels to identify valid edge center pixels in at least one row is accomplished on a row-by-row basis, and comprises the steps of applying a sized edge pixel filter to each pixel in a given row and determining, based on the filter output, whether each pixel is a valid edge center pixel, wherein the sized edge pixel filter is an array of numbers normalized to sum to 1, and wherein the width of the sized edge pixel filter has a width equal to the expected width in pixels of a vehicle image in the row being processed.

21. A method for identifying features in an electronic image, comprising the steps of:

using a camera for providing an electronic source image generated with an array of pixels;

using a processor for;

processing the electronic source image to identify edge pixels that make up edge segments of the electronic source image;

determining which of the identified edge pixels in at least one row are target edge pixels at the approximate horizontal center of a horizontal edge segment of essentially contiguous edge pixels having a target width;

determining symmetry values for test regions in the electronic source image, wherein the test regions are located relative to corresponding target edge pixels, and wherein size of the test regions is a function of the row in which the corresponding target edge pixels are located;

comparing determined symmetry values of test regions in the electronic source image that correspond to a segment of essentially contiguous edge pixels having a target width to identify the test region having the highest symmetry value; and

classifying test regions having the highest symmetry value to identify test regions of interest, wherein a test region of interest is a test region comprising image characteristics consistent with the presence of a type of object in the source image that is to be detected.

22. The method of claim 21, further comprising the step of refining the boundaries of identified test regions of interest to create refined regions of interest that more closely correspond in at least one of size and shape to objects to be detected.

23. The method of claim 22, further comprising the step of providing an indication of the location and size of the refined regions of interest to additional processing circuitry for processing.

24. A method for identifying potential vehicle regions in an electronic image, comprising the steps of:

using a camera for providing a source image generated by an array of pixels including at least one vehicle to be detected;

using a processor for;

electronically processing the source image to provide an edge image including edges corresponding to the edges of objects in the source image, wherein the edge image edges comprise edge pixels;

removing edges from the edge image that are angled by a predetermined amount relative to horizontal;

vertically thickening edges in the edge image;

removing edges from the edge image that are longer than a maximum length; and

identifying potential vehicle base edges in a row of the edge image, wherein a potential vehicle base edge is a horizontal edge of essentially contiguous edge pixels in the row, and wherein the horizontal edge has a width that is approximately equal to an expected width of a vehicle in the row.

25. The method of claim 24, further comprising the step of identifying at least one vehicle test region in the source image and determining a symmetry value for the at least one vehicle test region, wherein the at least one vehicle test region is rectangular in shape and has a bottom that corresponds in location and width to a potential vehicle base edge, and wherein the at least one vehicle test region has a height that is a function of the width of the potential vehicle base edge.

26. The method of claim 25, further comprising the step of comparing the symmetry value of the at least one vehicle test region to a symmetry value of at least one other vehicle test region in the same row to identify which of the compared vehicle test regions has the highest symmetry value.

27. The method of claim 26, further comprising the step of evaluating at least one characteristic of at least one vehicle test region having the highest symmetry value and classifying the at least one vehicle test region as a vehicle region of interest if the at least one characteristic meets at least one predetermined criteria indicative of a vehicle being present in the at least one vehicle test region.

28. The method of claim 27, further comprising the step of altering at least one of a border and location of a vehicle region of interest to provide a refined vehicle region of interest that more closely corresponds to at least one of the size and shape of vehicle in the source image.

29. The method of claim 28, further comprising the step of providing an indication of the location and size in the source image of a refined region of interest to additional processing circuitry configured to detect vehicles in the source image.