Method and apparatus for the recognition of obstacles

• US 8,155,380 B2
• Filed: 08/29/2008
• Issued: 04/10/2012
• Est. Priority Date: 09/24/2007
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Ma, Guanglin, Park, Ing Su-Birm, Ioffe, Alexander
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US12/231,550
• Publication Number:
  US 20090080702A1
• Time in Patent Office: 1,320 Days
• Field of Search: 382/103, 382/104, 340/435, 340/539.22, 340/539.26, 340/545.2, 340/547, 340/598, 340/619, 701/10, 701/201, 701/223, 701/226, 180/9.32, 434/68
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A method and apparatus for recognizing obstacles in an environment about a motorized vehicle moving along a roadway is presented. The apparatus includes an optical sensor and a data processing device. A method using the apparatus includes taking images with the optical sensor which contains at least a first and a second image. The first and the second image are each transformed above and below a plane as viewed by the optical sensor to further determine upper and lower difference images by the data processing device. The data processing device further determines whether an obstacle is located in the travel path of the vehicle by evaluation of the lower and upper difference images.

Claims

1. A method for recognition of an obstacle in an environment in view of an optical sensor attached to a vehicle, said environment including a travel path of the vehicle, comprising:

taking a first image by using the optical sensor at a first time and taking a second image at a later second time;

generating a first transformed lower image of the first image lying below the horizon by projecting an image section from an image plane of the optical sensor into a ground plane;

generating a first transformed upper image of the first image lying above the horizon by projecting an image section from the image plane of the optical sensor into a virtual plane parallel to the ground plane;

generating a second transformed lower image of the second image lying below the horizon by projecting an image section from the image plane of the optical sensor into the ground plane;

generating a second transformed upper image of the second image lying above the horizon by projecting the image section from the image plane of the optical sensor into the virtual plane parallel to the ground plane;

determining a lower difference image from the first and the second transformed lower part images;

determining an upper difference image from the first and the second upper part images; and

determining by evaluation of the lower difference image and the upper difference image whether the obstacle is located in the travel path of the vehicle.

2. The method in accordance with claim 1, wherein a respective object is classified as the obstacle and is transformed back into the image plane of the optical sensor.

3. The method in accordance with claim 1, wherein the optical sensor is a video camera.

4. The method in accordance with claim 3, wherein the optical sensor is a mono-camera.

5. The method in accordance with claim 1, wherein vehicle speed, change in vehicle travel direction, and time duration between the first and the second taking times are considered in the determination of the difference images from the respective first and second transformed images.

6. The method in accordance with claim 1, wherein vehicle movement is taken into account by translation and rotation of the first transformed lower image and of the second transformed lower image relative to one another and by translation and rotation of the first transformed upper image and of the second transformed upper part image relative to one another.

7. The method in accordance with claim 1, wherein said travel path includes the vehicle travelling along a road and the road has road irregularities, and said road irregularities are one of,

(i) vehicle inclination, and

(ii) vehicle vibration,

the method further includes,

compensating said road irregularities by image stabilization by an apparatus configured to perform said method, and said apparatus further includes said optical sensor.

8. A media including computer-readable instructions for recognition of an obstacle in an environment in view of an optical sensor attached to a vehicle using a first image taken by the optical sensor at a first time and a second image taken by the optical sensor at a later second time, said environment including a travel path of the vehicle, said computer-readable instructions being adapted to configure a data processing device to carry out a method comprising:

generating a first transformed lower image of the first image lying below the horizon by projecting an image section from an image plane of the optical sensor into a ground plane;

generating a first transformed upper image of the first image lying above the horizon by projecting an image section from the image plane of the optical sensor into a virtual plane parallel to the ground plane;

generating a second transformed lower image of the second image lying below the horizon by projecting an image section from the image plane of the optical sensor into the ground plane;

generating a second transformed upper image of the second image lying above the horizon by projecting the image section from the image plane of the optical sensor into the virtual plane parallel to the ground plane;

determining a lower difference image from the first and the second transformed lower part images;

determining an upper difference image from the first and the second upper part images; and

determining by evaluation of the lower difference image and the upper difference image whether the obstacle is located in the travel path of the vehicle.

9. An apparatus for recognition of an obstacle in an environment to a vehicle in which the environment includes a travel path of the vehicle, said apparatus comprising:

an optical sensor attached to a vehicle and having a view of an environment, said optical sensor taking a first image at a first time and a second image at a later second time; and

a data processing device adapted to receive the first image and the second image and configured to carry out a method comprising;

generating a first transformed lower image of the first image lying below the horizon by projecting an image section from an image plane of the optical sensor into a ground plane,

generating a first transformed upper image of the first image lying above the horizon by projecting an image section from the image plane of the optical sensor into a virtual plane parallel to the ground plane,

generating a second transformed lower image of the second image lying below the horizon by projecting an image section from the image plane of the optical sensor into the ground plane,

generating a second transformed upper image of the second image lying above the horizon by projecting the image section from the image plane of the optical sensor into the virtual plane parallel to the ground plane,

determining a lower difference image from the first and the second transformed lower part images,

determining an upper difference image from the first and the second upper part images, and

determining by evaluation of the lower difference image and the upper difference image whether the obstacle is located in the travel path of the vehicle.