IMAGE RECOGNITION SYSTEM FOR A VEHICLE AND CORRESPONDING METHOD

• US 20160104047A1
• Filed: 09/25/2015
• Published: 04/14/2016
• Est. Priority Date: 10/10/2014
• Status: Active Grant

• Current Assignee: Application Solutions (Electronics And Vision) Limited (Continental AG)
• Original Assignee: Application Solutions (Electronics And Vision) Limited (Continental AG)
• Inventors: Sibiryakov, Alexander
• Granted Patent: US 9,846,812 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An image recognition system and method for a vehicle, including at least two camera units, each being configured to record an image of a road in the vicinity of the vehicle and to provide image data representing the respective image of the road, a first image processor configured to combine the image data provided by the at least two camera units into a first top-view image. The first top-view image is aligned to a road image plane, a first feature extractor configured to extract lines from the first top-view image, a second feature extractor configured to extract an optical flow from the first top-view image and a second top-view image, generated before the first top-view image by the first image processor, and a kerb detector configured to detect kerbs in the road based on the extracted lines and the extracted optical flow and provide kerb data representing the detected kerbs.

Claims

1. An image recognition system for a vehicle comprising:

at least two camera units, each camera unit configured to record an image of a road in the vicinity of the vehicle and to provide image data representing the respective image of the road;

a first image processor configured to combine the image data provided by the at least two camera units into a first top-view image, wherein the first top-view image is aligned to a road image plane;

a first feature extractor configured to extract lines from the first top-view image;

a second feature extractor configured to extract an optical flow from the first top-view image and a second top-view image, which was generated before the first top-view image by the first image processor; and

a kerb detector configured to detect kerbs in the road based on the extracted lines and the extracted optical flow and provide kerb data representing the detected kerbs.

2. The image recognition system according to claim 1,

wherein the first feature extractor is configured to execute a line extraction algorithm.

3. The image recognition system according to claim 1,

wherein the second feature extractor is configured to extract the optical flow as a first set of optical flow vectors based on a sparse optical flow detection from the first top-view image and the second top-view image, using feature-matching or block-matching.

4. The image recognition system according to claim 3,

wherein the kerb detector is configured to separate the first set of optical flow vectors into a first subset of optical flow vectors and a second subset of optical flow vectors for every detected line;

wherein for each detected line the first subset of optical flow vectors represents optical flow vectors, which are located on the same side of the respective detected line as the vehicle; and

wherein for each detected line the second subset of optical flow vectors represents optical flow vectors, which are located on the opposite side of the respective detected line as the vehicle.

5. The image recognition system according to claim 4, wherein the kerb detector is configured to test for every detected line whether the respective first subset of optical flow vectors and the respective second subset of optical flow vectors belong to the same surface and detect a kerb where the respective first subset of optical flow vectors and the respective second subset of optical flow vectors do not belong to the same surface.

6. The image recognition system according to claim 5, wherein the kerb detector is configured to detect and delete outliers in the first subset of optical flow vectors and the second subset of optical flow vectors prior to testing the first subset of optical flow vectors and the second subset of optical flow vectors.

7. The image recognition system according to claim 5,

wherein the kerb detector comprises a motion model calculator, which is configured to calculate at least a fitted global motion model comprising a rotation and a translation, based on the first set of optical flow vectors, and based on the inline vectors of the first set of optical flow vectors;

and

wherein the kerb detector is configured to calculate based on the at least one fitted global motion model for every detected line a first noise distribution for the respective first subset and a second noise distribution for the respective second subset and detect a kerb at the position of the respective line if the difference between the first noise distribution and the second noise distribution exceeds a predetermined threshold.

8. The image recognition system according to claim 7, wherein the kerb detector comprises a memory configured to store the positions of the detected kerbs in different moments of time and the respective fitted global motion models;

and

wherein the kerb detector is configured to combine the stored positions of the detected kerbs and the respective fitted global motion models to provide a kerb detection with a stronger evidence of the respective kerb.

9. The image recognition system according to claim 1,

comprising only one single camera unit;

wherein the first image processor is configured to generate the first top-view image based on two consecutive sets of image data which are based on two images consecutively recorded by the single camera unit while the vehicle is in motion.

16. The image recognition system according to claim 2, wherein the line extraction algorithm is a Hough transform algorithm.

17. The image recognition system according to claim 6, wherein the kerb detector comprises a motion model calculator, which is configured to calculate at least a fitted global motion model comprising a rotation and a translation, based on the first set of optical flow vectors, based on the inline vectors of the first set of optical flow vectors;

and

wherein the kerb detector is configured to calculate based on the at least one fitted global motion model for every detected line a first noise distribution for the respective first subset and a second noise distribution for the respective second subset and detect a kerb at the position of the respective line if the difference between the first noise distribution and the second noise distribution exceeds a predetermined threshold.

10. A method for recognising images in a vehicle, comprising:

recording with at least a first camera unit and a second camera unit at least two different images of a road in the vicinity of the vehicle and providing image data representing the respective images of the road;

combining the provided image data into a first top-view image, wherein the first top-view image is aligned to a road image plane;

extracting lines from the first top-view image;

extracting an optical flow from the first top-view image and a second top-view image, which was generated before the first top-view image;

detecting kerbs in the road based on the extracted lines and the extracted optical flow; and

providing kerb data representing the detected kerbs.

11. The method according to claim 10, wherein extracting lines comprises executing a line extraction algorithm;

and/or

wherein extracting an optical flow comprises extracting the optical flow as a first set of optical flow vectors based on a sparse optical flow detection from the first top-view image and the second top-view image using feature-matching or block-matching.

12. The method according to claim 11,

wherein detecting kerbs comprises separating the first set of optical flow vectors into a first subset of optical flow vectors and a second subset of optical flow vectors for every detected line;

wherein for each detected line the first subsets of optical flow vectors represents optical flow vectors, which are on the same side of the respective detected line as the vehicle; and

wherein for each detected line the second subset of optical flow vectors represents optical flow vectors, which are on the opposite side of the respective detected line as the vehicle.

13. The method according to claim 12, wherein detecting kerbs comprises testing for every detected line whether the respective first subset of optical flow vectors and the respective second subset of optical flow vectors belong to the same surface and detecting a kerb where the respective first subset of optical flow vectors and the respective second subset of optical flow vectors do not belong to the same surface;

and

wherein detecting kerbs comprises detecting and deleting outliers in the first subset of optical flow vectors and the second subset of optical flow vectors prior to testing the first subset of optical flow vectors and the second subset of optical flow vectors.

14. The method according to claim 12,

wherein detecting kerbs comprises calculating at least a fitted global motion model, especially comprising a rotation and a translation, based on the first set of optical flow vectors, especially based on the inline vectors of the first set of optical flow vectors;

and

wherein detecting kerbs comprises calculating for every detected line a first noise distribution for the respective first subset and a second noise distribution for the respective second subset and detecting a kerb at the position of the respective line if the difference between the first noise distribution and the second noise distribution exceeds a predetermined threshold.

15. The method according to claim 14, wherein detecting kerbs comprises storing the positions of the detected kerbs in different moments of time and the respective fitted global motion models;

and

wherein detecting kerbs comprises combining the stored positions of the detected kerbs and the respective fitted global motion models to provide a kerb detection with a stronger evidence of the respective kerb.

18. The method according to claim 11 a Hough transform wherein the line extraction algorithm is a Hough transform algorithm.

19. The method according to claim 13,

wherein detecting kerbs comprises calculating at least a fitted global motion model, especially comprising a rotation and a translation, based on the first set of optical flow vectors, especially based on the inline vectors of the first set of optical flow vectors;

and

wherein detecting kerbs comprises calculating for every detected line a first noise distribution for the respective first subset and a second noise distribution for the respective second subset and detecting a kerb at the position of the respective line if the difference between the first noise distribution and the second noise distribution exceeds a predetermined threshold.