STEREOSCOPIC CAMERA OBJECT DETECTION SYSTEM AND METHOD OF ALIGNING THE SAME

• US 20140168377A1
• Filed: 12/13/2012
• Published: 06/19/2014
• Est. Priority Date: 12/13/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Delphi Technologies, Inc. (BorgWarner Incorporated)
• Inventors: Cluff, Charles A., Taylor, Ronald M.
• Granted Patent: US 9,066,085 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

A system, controller, and method for aligning a stereo camera of a vehicle mounted object detection system that includes a first camera and a second camera mounted spaced apart on a vehicle. An image from each camera at two different times is used to determine an observed displacement of an object relative to the vehicle. A predicted displacement of the object relative to the vehicle is also determined using either a difference of vehicle position measured based on other vehicle measurements or GPS, or a difference of size of the object in images taken at the two different times. Alignment is provided by determining a triangulation correction based on a difference of the observed displacement and the predicted displacement to correct for misalignment of the cameras.

Claims

1. A method for aligning a stereo camera of a vehicle mounted object detection system, said system comprising a first camera and a second camera mounted spaced apart on a vehicle, said method comprising:

determining a first position of an object in a first image from the first camera captured at a first time;

determining a second position of the object in a second image from the second camera captured at the first time;

determining an initial location of the object relative to the vehicle at the first time based on triangulation of the first image and the second image;

determining a third position of the object in a third image from the first camera captured at a second time distinct from the first time;

determining a fourth position of the object in a fourth image from the second camera captured at the second time;

determining a subsequent location of the object relative to the vehicle at the second time based on triangulation of the third image and the fourth image;

determining an observed displacement of the object relative to the vehicle based on a difference of the initial location and the subsequent location;

determining a predicted displacement of the object relative to the vehicle based on one of

a) a difference of vehicle position at the first time and the second time, and

b) a difference of an initial size of the object in the first image and a subsequent size of the object in the third image; and

determining a triangulation correction based on a difference of the observed displacement and the predicted displacement to correct for misalignment of the first camera and the second camera when determining object location based on triangulation of images from the first camera and the second camera.

2. The method in accordance with claim 1, wherein the step of determining a predicted displacement of the object relative to the vehicle is based on both a) the difference of vehicle position at the first time and the second time, and b) the difference of an initial size of the object in the first image and a subsequent size of the object in the third image.

3. The method in accordance with claim 1, wherein the method includes mounting the first camera spaced apart from the second camera by at least five-hundred millimeters (500 mm).

4. The method in accordance with claim 1, wherein the method includes determining that the initial size is greater than a size threshold prior to determining the initial location of the object.

5. The method in accordance with claim 1, wherein the method includes determining if the first position of the object intersects with an boundary of the first image, and searching for a new object if the object intersects with the boundary.

6. The method in accordance with claim 1, wherein the method includes tracking the object in images from the first camera and the second camera after the first time.

7. The method in accordance with claim 6, wherein the method includes determining the second time if tracking the object indicates that relative displacement of the object and the vehicle is greater than an object displacement threshold.

8. The method in accordance with claim 1, wherein the method includes monitoring vehicle displacement after the first time.

9. The method in accordance with claim 8, wherein the method includes determining the second time if monitoring the vehicle displacement indicates that the vehicle displacement is greater than a vehicle displacement threshold.

10. An object detection system for detecting an object proximate to a vehicle, said system comprising:

a first camera configured to be mounted on the vehicle;

a second camera configured to be mounted on the vehicle spaced apart from the first camera; and

a controller configured to receive images from the first camera and the second camera, said controller programmed to

determine a first position of an object in a first image from the first camera captured at a first time;

determine a second position of the object in a second image from the second camera captured at the first time;

determine an initial location of the object relative to the vehicle at the first time based on triangulation of the first image and the second image;

determine a third position of the object in a third image from the first camera captured at a second time distinct from the first time;

determine a fourth position of the object in a fourth image from the second camera captured at the second time;

determine a subsequent location of the object relative to the vehicle at the second time based on triangulation of the third image and the fourth image;

determine an observed displacement of the object relative to the vehicle based on a difference of the initial location and the subsequent location;

determine a predicted displacement of the object relative to the vehicle based on one of

a) a difference of vehicle position at the first time and the second time, and

b) a difference of an initial size of the object in the first image and a subsequent size of the object in the third image; and

determine a triangulation correction based on a difference of the observed displacement and the predicted displacement to correct for misalignment of the first camera and the second camera when determine object location based on triangulation of images from the first camera and the second camera.

11. A controller for an object detection system for detecting an object proximate to a vehicle, said controller comprising:

a first input configured to receive a first signal from a first camera mounted on the vehicle;

a second input configured to receive a second signal from a second camera mounted on the vehicle spaced apart from the first camera; and

a processor configured to

receive images from the first camera and the second camera;

determine a first position of an object in a first image from the first camera captured at a first time;

determine a second position of the object in a second image from the second camera captured at the first time;

determine an initial location of the object relative to the vehicle at the first time based on triangulation of the first image and the second image;

determine a third position of the object in a third image from the first camera captured at a second time distinct from the first time;

determine a fourth position of the object in a fourth image from the second camera captured at the second time;

determine a subsequent location of the object relative to the vehicle at the second time based on triangulation of the third image and the fourth image;

determine an observed displacement of the object relative to the vehicle based on a difference of the initial location and the subsequent location;

determine a predicted displacement of the object relative to the vehicle based on one of

a) a difference of vehicle position at the first time and the second time, and

b) a difference of an initial size of the object in the first image and a subsequent size of the object in the third image; and

determine a triangulation correction based on a difference of the observed displacement and the predicted displacement to correct for misalignment of the first camera and the second camera when determine object location based on triangulation of images from the first camera and the second camera.