WIDE BASELINE OBJECT DETECTION STEREO SYSTEM

• US 20160086040A1
• Filed: 09/19/2014
• Published: 03/24/2016
• Est. Priority Date: 09/19/2014
• Status: Active Grant

• Current Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Original Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Inventors: Kuehnle, Andreas U., Molin, Hans M., Gyori, Marton, Boon, Cathy L., Szoellosi, Adam
• Granted Patent: US 9,990,550 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

When detecting an object of interest, such as a bicyclist passing a truck, two downward looking cameras both capture images of the cyclist and detect the cyclist as a deviation from the flat ground plane. The ground plane is reconstructed using a homography (projection) matrix of each camera and compared. Where the camera images do not agree, the ground is not flat. The cyclist is located as the intersection of the rays extending to either end of the area of disagreement between the images.

Claims

1. A system that facilitates detecting an object in a surround-view image of a vehicle, comprising:

a first camera that captures a first image of an object;

a second camera that captures a second image of the object;

a processor configured to;

generate a composite image by merging the first and second images;

analyze the composite image to detect one or more features indicative of the type of object in the composite image; and

generate a signal that indicates the presence of the object and location of the object relative to the vehicle.

2. The system according to claim 1, further comprising a disagreement calculation module configured to analyze the composite image to identify an area of disagreement D between the first and second images.

3. The system according to claim 2, further comprising a ray calculation module configured to calculate at least one ray emanating from each camera through its field of view to a boundary of a more distant half, relative to each camera, of the area of disagreement D.

4. The system according to claim 3, further comprising a ray intersection identification module configured to identify an intersection R of camera rays, wherein the intersection corresponds to a highest point of the object, and wherein the processor is configured to execute a tracking module configured to track the highest point of the object.

5. The method according to claim 4, wherein the object is a cyclist and wherein the processor is configured to execute a tracking module configured to track the highest point of the cyclist.

6. The system according to claim 1, wherein the processor is further configured to execute an object shape identification module that identifies a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist determined to be positioned where the front and rear view representations intersect in the composite image.

7. The system according to claim 6, wherein the processor is further configured to determine a height of the head of the cyclist above the ground in the composite image, and wherein the processor is configured to execute a tracking filter that tracks the location of the head of the cyclist.

8. The system according to claim 1, wherein the processor is further configured to execute a wheel identification module that is configured to detect one of more wheels of a bicycle.

9. The system according to claim 8, wherein the processor is further configured to execute a Hough transform to detect one or more ellipses corresponding to the wheels of the bicycle in each of a front view and a rear view in the composite image.

10. The system according to claim 9, wherein the processor is further configured to track the detected ellipses in order to track the bicycle.

11. A method of detecting and tracking an object, comprising:

capturing a first image, from a first vantage point, of an object;

capturing a second image, from a second vantage point, of the object;

generating a composite image by merging the first and second images;

analyzing the composite image to detect one or more features in the composite image; and

generating a signal that indicates the presence of the object and location of the object relative to the vehicle.

12. The method according to claim 11, further comprising:

analyzing the composite image to identify an area of disagreement D between the first and second images;

calculating at least one ray emanating from each camera through its field of view to a boundaries of a more distant half, relative to each camera, of the area of disagreement D; and

identifying an intersection R of camera rays, wherein the intersection corresponds to a highest point of the object, and tracking the highest point of the object.

13. The method according to claim 11, further comprising identifying a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist and determined to be positioned where the front and rear view representations intersect in the composite image.

14. The method according to claim 13, further comprising determining a height of the head of the cyclist above the ground in the composite image, and tracking the location of the head of the cyclist.

15. The method according to claim 11, further comprising detecting one or more ellipses corresponding to the wheels of a bicycle in each of a front view and a rear view of the object in the composite image.

16. The method according to claim 15, further comprising tracking the detected ellipses in order to track the bicycle.

17. A processor configured to execute computer-executable instructions for detecting and tracking an object, the instructions comprising:

capturing a first image, from a first vantage point, of an object;

capturing a second image, from a second vantage point, of the object;

generating a composite image by merging the first and second images;

analyzing the composite image to detect one or more features indicative of the type of object in the composite image; and

generating a signal that indicates the presence of the object and location of the object relative to the vehicle.

18. The processor according to claim 17, the instructions further comprising:

analyzing the composite image to identify an area of disagreement D between the first and second images;

calculating at least one ray emanating from each camera through its field of view to a boundaries of a more distant half, relative to each camera, of the area of disagreement D; and

identifying an intersection R of camera rays, wherein the intersection corresponds to a highest point of the object, and tracking the highest point of the object.

19. The processor according to claim 17, the instructions further comprising identifying a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist determined to be positioned where the front and rear view representations intersect in the composite image.

20. The processor according to claim 19, the instructions further comprising determining a height of the head of the cyclist above the ground in the composite image, and tracking the location of the head of the cyclist.

21. The processor according to claim 17, the instructions further comprising detecting one or more ellipses corresponding to wheels of a bicycle in each of a front view and a rear view of the object in the composite image.

22. The processor according to claim 21, the instructions further comprising tracking the detected ellipses in order to track the bicycle.

23. An apparatus that facilitates detecting and tracking an object as it moves relative to a vehicle, comprising:

means for capturing a first image, from a first vantage point, of an object;

means for capturing a second image, from a second vantage point, of the object;

means for generating a composite image by merging the first and second images;

means for analyzing the composite image to detect one or more features indicative of the type of object in the composite image; and

means for generating a signal that indicates the presence of the object and location of the object relative to the vehicle.