AUTOMATIC ORIENTATION ESTIMATION OF CAMERA SYSTEM RELATIVE TO VEHICLE

• US 20160284087A1
• Filed: 03/24/2015
• Published: 09/29/2016
• Est. Priority Date: 03/24/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Natroshvili, Koba, Scholl, Kay-Ulrich
• Granted Patent: US 10,013,761 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Methods, apparatuses, and systems may provide for using the motion of a vehicle to estimate the orientation of a camera system of a vehicle relative to the vehicle. Image data may be received from a plurality of cameras positioned on the vehicle, and a first constraint set may be determined for the plurality of cameras based on a plurality of feature points in a ground plane proximate to the vehicle. A second constraint set may be determined based on one or more borders of the vehicle. One or more of the cameras may be automatically calibrated based on the first constraint set and the second constraint set.

Claims

1. A camera-based driver-assistance system comprising:

a plurality of cameras each comprising an electro-optical component to capture images, wherein the cameras are located in different areas on a vehicle hosting the camera-based driver-assistance system;

a feature point analyzer to determine, for the plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to the vehicle;

a border analyzer to determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle;

a calibrator to conduct an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set; and

a display to visually output a result of the automatic calibration.

2. The system of claim 1, wherein the border analyzer includes:

an image data interface to receive image data associated with the plurality of cameras;

a frame analyzer to determine an optical flow based on a plurality of frames in the image data;

a border detector to identify the one or more borders based on the optical flow.

3. The system of claim 1, further including a sensor interface to receive additional sensor data from one or more sensors on the vehicle, wherein the border analyzer is to determine the second constraint set based on the additional sensor data and the additional sensor data is to include one or more of velocity data, steering angle data, wheel information, and inertia data.

4. The system of claim 1, wherein the feature point analyzer is to include:

an orientation estimator to estimate an orientation of each feature point relative to a coordinate system of the plurality of cameras; and

a location estimator to estimate a location of each feature point relative to the coordinate system of the plurality of cameras.

5. The system of claim 1, further including a feature point extractor to receive image data associated with the plurality of cameras, and detect the plurality of feature points based on the image data.

6. The system of claim 1, wherein the calibrator is to estimate an orientation of one or more of the plurality of cameras to conduct the automatic calibration.

7. An apparatus comprising:

a feature point analyzer to determine, for the plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to the vehicle;

a border analyzer to determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and

a calibrator to conduct an automatic calibration one or more of the plurality of cameras based on the first constraint set and the second constraint set.

8. The apparatus of claim 7, wherein the border analyzer includes:

an image data interface to receive image data associated with the plurality of cameras;

a frame analyzer to determine an optical flow based on a plurality of frames in the image data; and

a border detector to identify the one or more borders based on the optical flow.

9. The apparatus of claim 7, further including a sensor interface to receive additional sensor data from one or more sensors on the vehicle, wherein the border analyzer is to determine the second constraint set based on the additional sensor data and the additional sensor data is to include one or more of velocity data, steering angle data, wheel information, and inertia data.

10. The apparatus of claim 7, wherein the frame analyzer includes:

an orientation estimator to estimate an orientation of each feature point relative to a coordinate system of the plurality of cameras; and

a location estimator to estimate a location of each feature point relative to the coordinate system of the plurality of cameras.

11. The apparatus of claim 7, further including a feature point extractor to receive image data associated with the plurality of cameras, and detect the plurality of feature points based on the received image data.

12. The apparatus of claim 7, wherein the calibrator is to estimate an orientation of one or more of the plurality of cameras to conduct the automatic calibration.

13. A method comprising:

determining, for a plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to a vehicle;

determining, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and

conducting an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set.

14. The method of claim 13, wherein determining the second constraint set includes:

receiving image data associated with the plurality of cameras;

determining an optical flow based on a plurality of frames in the image data; and

identifying the one or more borders based on the optical flow.

15. The method of claim 13, further including receiving additional sensor data from one or more sensors on the vehicle, wherein the second constraint set is determined based on the additional sensor data and the additional sensor data includes one or more of velocity data, steering angle data, wheel information, and inertia data.

16. The method of claim 13, wherein determining the first constraint set includes:

estimating an orientation of each feature point relative to a coordinate system of the plurality of cameras; and

estimating a location of each feature point relative to the coordinate system of the plurality of cameras.

17. The method of claim 13, further including:

receiving image data associated with the plurality of cameras; and

detecting the plurality of feature points based on the image data.

18. The method of claim 13, wherein conducting the automatic calibration includes estimating an orientation of one or more of the plurality of cameras.

19. At least one non-transitory computer readable storage medium, comprising a set of instructions which, when executed by a computing device causes the computing device to:

determine, for a plurality of cameras, a first constraint set based on a plurality of feature points in a ground plane proximate to a vehicle;

determine, for the plurality of cameras, a second constraint set based on one or more borders of the vehicle; and

conduct an automatic calibration of one or more of the plurality of cameras based on the first constraint set and the second constraint set.

20. The at least one computer readable storage medium of claim 19, wherein the instructions, when executed, cause the computing device to:

receive image data associated with the plurality of cameras;

determine an optical flow based on a plurality of frames in the image data;

identify the one or more borders based on the optical flow to determine the second constraint set.

21. The at least one computer readable storage medium of claim 19, wherein the instructions, when executed, cause the computing device to receive additional sensor data from one or more sensors on the vehicle, wherein the second constraint set is determined based on the additional sensor data and the additional sensor data includes one or more of velocity data, steering angle data, wheel information, inertia data.

22. The at least one computer readable storage medium of claim 19, wherein the instructions, when executed, cause the computing device to:

estimate an orientation of each feature point relative to a coordinate system of the plurality of cameras; and

estimate a location of each feature point relative to the coordinate system of the plurality of cameras to determine the first constraint set.

23. The at least one computer readable storage medium of claim 19, wherein the instructions, when executed, cause the computing device to:

receive image data associated with the plurality of cameras, and

detect the plurality of feature points based on the image data.

24. The at least one computer readable storage medium of claim 19, wherein the instructions, when executed, cause the computing device to estimate an orientation of one or more of the plurality of cameras to conduct the automatic calibration.