Wide baseline object detection stereo system

• US 9,990,550 B2
• Filed: 09/19/2014
• Issued: 06/05/2018
• Est. Priority Date: 09/19/2014
• Status: Active Grant

• Current Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Original Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Inventors: Kuehnle, Andreas U., Molin, Hans M., Gyori, Marton, Szoellosi, Adam, Boon, Cathy L.
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US14/491,009
• Publication Number:
  US 20160086040A1
• Time in Patent Office: 1,355 Days
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

When detecting an object of interest, such as a bicyclist passing a truck, two downward looking cameras both capture images of the cyclist and detect the cyclist as a deviation from the flat ground plane. The ground plane is reconstructed using a homography (projection) matrix of each camera and compared. Where the camera images do not agree, the ground is not flat. The cyclist is located as the intersection of the rays extending to either end of the area of disagreement between the images.

Claims

1. A system that facilitates detecting an object in a surround-view image of a vehicle, comprising:

a first camera that captures a first image of an object;

a second camera that captures a second image of the object;

a processor configured to;

generate a composite image by merging the first and second images;

analyze the composite image to detect one or more features indicative of the type of object in the composite image; and

generate a signal that indicates the presence of the object and location of the object relative to the vehicle; and

a ray calculation module configured to calculate at least one ray emanating from each camera through its field of view to a boundary of a more distant half, relative to each camera, of an area of disagreement D between the first and second images, wherein the calculated rays define an outline of the object;

wherein the processor is further configured to scan pixels defining the area of disagreement in order to determine where the calculated rays intersect, and identify a highest point of intersection as a highest point of the object;

wherein the processor is further configured to identify a V-shape formed in the composite image by the front and rear views of the object, and to confirm that the object is a one of cyclist or a pedestrian.

2. The system according to claim 1, further comprising a disagreement calculation module configured to analyze the composite image to identify the area of disagreement D between the first and second images.

3. The system according to claim 1, further comprising a ray intersection identification module configured to identify an intersection R of camera rays, wherein the intersection corresponds to the highest point of the object, and wherein the processor is configured to execute a tracking module configured to track the highest point of the object.

4. The method according to claim 3, wherein the object is a cyclist and wherein the processor is configured to execute a tracking module configured to track the highest point of the cyclist.

5. The system according to claim 1, wherein the processor is further configured to execute an object shape identification module that identifies a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist determined to be positioned where the front and rear view representations intersect in the composite image.

6. The system according to claim 5, wherein the processor is further configured to determine a height of the head of the cyclist above the ground in the composite image, and wherein the processor is configured to execute a tracking filter that tracks the location of the head of the cyclist.

7. The system according to claim 1, wherein the processor is further configured to execute a wheel identification module that is configured to detect one of more wheels of a bicycle.

8. The system according to claim 7, wherein the processor is further configured to execute a Hough transform to detect one or more ellipses corresponding to the wheels of the bicycle in each of a front view and a rear view in the composite image.

9. The system according to claim 8, wherein the processor is further configured to track the detected ellipses in order to track the bicycle.

10. The system according to claim 1, further comprising an alert generator that generates an alert signal upon detection of the object, and wherein the processor is further configured to receive the alert signal and generate a visual alert on the surround-view image to draw the driver'"'"'s attention to the object.

12. The method according to claim 10, further comprising:

analyzing the composite image to identify the area of disagreement D between the first and second images; and

identifying an intersection R of camera rays, wherein the intersection corresponds to a highest point of the object, and tracking the highest point of the object.

13. The method according to claim 10, further comprising identifying a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist and determined to be positioned where the front and rear view representations intersect in the composite image.

14. The method according to claim 13, further comprising determining a height of the head of the cyclist above the ground in the composite image, and tracking the location of the head of the cyclist.

15. The method according to claim 10, further comprising detecting one or more ellipses corresponding to the wheels of a bicycle in each of a front view and a rear view of the object in the composite image.

16. The method according to claim 15, further comprising tracking the detected ellipses in order to track the bicycle.

11. A method of detecting and tracking an object, comprising:

capturing a first image, from a first vantage point, of an object;

capturing a second image, from a second vantage point, of the object;

generating a composite image by merging the first and second images;

analyzing the composite image to detect one or more features in the composite image;

generating a signal that indicates the presence of the object and location of the object relative to the vehicle;

calculating at least one ray emanating from each camera through its field of view to a boundary of a more distant half, relative to each camera, of an area of disagreement D between the first and second images, wherein the calculated rays define an outline of the object;

scanning pixels defining the area of disagreement, determining where the calculated rays intersect, and identifying a highest point of intersection as a highest point of the object; and

identifying a V-shape formed in the composite image by the front and rear views of the object, and confirming that the object is a one of cyclist or a pedestrian.

17. A processor configured to detect and track an object, wherein the processor comprises:

circuitry configured to;

receive a first image, from a first vantage point, of an object;

receive a second image, from a second vantage point, of the object;

generate a composite image by merging the first and second images;

analyze the composite image to detect one or more features indicative of the type of object in the composite image;

generate a signal that indicates the presence of the object and location of the object relative to the vehicle; and

calculate at least one ray emanating from each camera through its field of view to a boundary of a more distant half, relative to each camera, of an area of disagreement D between the first and second images, wherein the calculated rays define an outline of the object;

wherein the circuitry is further configured to scan pixels defining the area of disagreement, determine where the calculated rays intersect, and identify a highest point of intersection as a highest point of the object; and

identify a V-shape formed in the composite image by the front and rear views of the object, and confirm that the object is a one of cyclist or a pedestrian.

18. The processor according to claim 17, wherein the circuitry is further configured to:

analyze the composite image to identify the area of disagreement D between the first and second images; and

identify an intersection R of camera rays, wherein the intersection corresponds to a highest point of the object, and tracking the highest point of the object.

19. The processor according to claim 17, wherein the circuitry is further configured to identify a shape formed by each of a front view representation and a rear view representation of the object in the composite image, wherein the object is a cyclist determined to be positioned where the front and rear view representations intersect in the composite image.

20. The processor according to claim 19, wherein the circuitry is further configured to determine a height of the head of the cyclist above the ground in the composite image, and tracking the location of the head of the cyclist.

21. The processor according to claim 17, wherein the circuitry is further configured to detect one or more ellipses corresponding to wheels of a bicycle in each of a front view and a rear view of the object in the composite image.

22. The processor according to claim 21, wherein the circuitry is further configured to track the detected ellipses in order to track the bicycle.

23. An apparatus that facilitates detecting and tracking an object as it moves relative to a vehicle, comprising:

means for capturing a first image, from a first vantage point, of an object;

means for capturing a second image, from a second vantage point, of the object;

means for generating a composite image by merging the first and second images;

means for analyzing the composite image to detect one or more features indicative of the type of object in the composite image;

means for generating a signal that indicates the presence of the object and location of the object relative to the vehicle;

means for calculating at least one ray emanating from each camera through its field of view to a boundary of a more distant half, relative to each camera, of an area of disagreement D between the first and second images, wherein the calculated rays define an outline of the object;

means for scanning pixels defining the area of disagreement, determining where the calculated rays intersect, and identifying a highest point of intersection as a highest point of the object; and

means for identifying a V-shape formed in the composite image by the front and rear views of the object and confirming that the object is a one of cyclist or a pedestrian.