Method for determining alignment of vehicular cameras

• US 10,868,974 B2
• Filed: 02/19/2018
• Issued: 12/15/2020
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lu, Yuesheng
• Primary Examiner(s): Kalapodas, Dramos
• Application Number:
  US15/899,105
• Publication Number:
  US 20180176482A1
• Time in Patent Office: 1,030 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

A method for determining alignment of cameras at a vehicle includes mounting first and second cameras at a vehicle and providing an image processor at the vehicle. While the vehicle is moving along the road, first image data captured via the first camera is processed to determine movement of a road feature and second image data captured by the second camera is processed to determine movement of the road feature when the road feature is present in the second image data. The determined movement of the road feature in first image data is compared to the determined movement of the road feature in second image data, and an offset of the second camera relative to the first camera is determined, with the offset caused by a rotational or translational change in position of the first or second camera relative to the other.

Claims

1. A method for determining alignment of cameras at a vehicle relative to one another, said method comprising:

mounting a first camera at a vehicle so as to have a first field of view encompassing a road surface of a road along which the vehicle is traveling;

mounting a second camera at the vehicle so as to have a second field of view encompassing the road surface of the road along which the vehicle is traveling;

wherein the first and second fields of view of the first and second cameras overlap partially;

providing an image processor at the vehicle;

while the vehicle is moving along the road, capturing first image data via the first camera;

while the vehicle is moving along the road, capturing second image data via the second camera;

providing first image data captured by the first camera to the image processor;

providing second image data captured by the second camera to the image processor;

processing, via the image processor, first image data captured by the first camera;

processing, via the image processor, second image data captured by the second camera;

dewarping the first image data and the second image data;

while the vehicle is moving along the road, determining, via processing of first image data captured by the first camera, movement of a road feature present in first image data captured by the first camera;

while the vehicle is moving along the road, determining, via processing of second image data captured by the second camera, movement of the road feature when the road feature is present in second image data captured by the second camera;

comparing the determined movement of the road feature in first image data captured by the first camera to the determined movement of the road feature in second image data captured by the second camera while the vehicle is moving along the road;

determining, via comparing the determined movements of the road feature, a rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle; and

responsive to determination of the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle, remapping the first and second image data based at least in part on the determined rotational and translational offset to at least partially accommodate misalignment of the second camera relative to the first camera.

2. The method of claim 1, wherein the first camera is mounted at the vehicle so as to have a sideward field of view, and wherein the second camera is mounted at the vehicle so as to have a rearward field of view.

3. The method of claim 1, wherein the first camera is mounted at the vehicle so as to have a forward field of view, and wherein the second camera is mounted at the vehicle so as to have a sideward field of view.

4. The method of claim 3, comprising mounting a third camera at the vehicle so as to have a rearward field of view.

5. The method of claim 4, comprising, while the vehicle is moving along the road, (i) determining, via processing of captured image data, movement of the road feature when the road feature is present in third image data captured by the third camera, (ii) comparing the determined movement of the road feature in second image data captured by the second camera to the determined movement of the road feature in third image data captured by the third camera, and (iii) determining, via comparing the determined movements of the road feature, a rotational and translational offset of the third camera at the vehicle relative to the second camera at the vehicle.

6. The method of claim 1, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a horizontal shift of the road feature while the vehicle is moving along the road.

7. The method of claim 1, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a vertical shift of the road feature in the second image data captured by the second camera as compared to the first image data captured by the first camera while the vehicle is moving along the road.

8. The method of claim 1, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a rotation of the road feature in the second image data captured by the second camera as compared to the first image data captured by the first camera while the vehicle is moving along the road.

9. The method of claim 1, comprising displaying video images derived from image data captured by the first and second cameras.

10. A method for determining alignment of cameras at a vehicle relative to one another, said method comprising:

mounting a first camera at a vehicle so as to have a first field of view encompassing a road surface of a road along which the vehicle is traveling;

mounting a second camera at the vehicle so as to have a second field of view encompassing the road surface of the road along which the vehicle is traveling;

wherein the first camera is mounted at the vehicle so as to have a sideward field of view, and wherein the second camera is mounted at the vehicle so as to have a rearward field of view;

wherein the first and second fields of view of the first and second cameras overlap partially;

providing an image processor at the vehicle;

while the vehicle is moving along the road, capturing first image data via the first camera;

while the vehicle is moving along the road, capturing second image data via the second camera;

providing first image data captured by the first camera to the image processor;

providing second image data captured by the second camera to the image processor;

processing, via the image processor, first image data captured by the first camera;

processing, via the image processor, second image data captured by the second camera;

dewarping the first image data and the second image data;

while the vehicle is moving along the road, determining, via processing of first image data captured by the first camera, movement of a road feature present in first image data captured by the first camera;

while the vehicle is moving along the road, determining, via processing of second image data captured by the second camera, movement of the road feature when the road feature is present in second image data captured by the second camera;

comparing the determined movement of the road feature in first image data captured by the first camera to the determined movement of the road feature in second image data captured by the second camera while the vehicle is moving along the road;

determining, via comparing the determined movements of the road feature, a rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle;

responsive to determination of the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle, remapping the first and second image data based at least in part on the determined rotational and translational offset to at least partially accommodate misalignment of the second camera relative to the first camera; and

displaying video images derived from image data captured by the first and second cameras.

11. The method of claim 10, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a horizontal shift of the road feature while the vehicle is moving along the road.

12. The method of claim 10, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a vertical shift of the road feature while the vehicle is moving along the road.

13. The method of claim 10, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a rotation of the road feature while the vehicle is moving along the road.

14. A method for determining alignment of cameras at a vehicle relative to one another, said method comprising:

mounting a first camera at a vehicle so as to have a first field of view encompassing a road surface of a road along which the vehicle is traveling;

mounting a second camera at the vehicle so as to have a second field of view encompassing the road surface of the road along which the vehicle is traveling;

wherein the first camera is mounted at the vehicle so as to have a forward field of view, and wherein the second camera is mounted at the vehicle so as to have a sideward field of view;

wherein the first and second fields of view of the first and second cameras overlap partially;

providing an image processor at the vehicle;

while the vehicle is moving along the road, capturing first image data via the first camera;

while the vehicle is moving along the road, capturing second image data via the second camera;

providing first image data captured by the first camera to the image processor;

providing second image data captured by the second camera to the image processor;

processing, via the image processor, first image data captured by the first camera;

processing, via the image processor, second image data captured by the second camera;

dewarping the first image data and the second image data;

while the vehicle is moving along the road, determining, via processing of first image data captured by the first camera, movement of a road feature present in first image data captured by the first camera;

while the vehicle is moving along the road, determining, via processing of second image data captured by the second camera, movement of the road feature when the road feature is present in second image data captured by the second camera;

comparing the determined movement of the road feature in first image data captured by the first camera to the determined movement of the road feature in second image data captured by the second camera while the vehicle is moving along the road;

determining, via comparing the determined movements of the road feature, a rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle;

wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a rotation of the road feature while the vehicle is moving along the road;

responsive to determination of the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle, remapping the first and second image data based at least in part on the determined rotational and translational offset to at least partially accommodate misalignment of the second camera relative to the first camera; and

displaying video images derived from image data captured by the first and second cameras.

15. The method of claim 14, comprising mounting a third camera at the vehicle so as to have a rearward field of view.

16. The method of claim 15, comprising, while the vehicle is moving along the road, (i) determining, via processing of captured image data, movement of the road feature when the road feature is present in third image data captured by the third camera, (ii) comparing the determined movement of the road feature in second image data captured by the second camera to the determined movement of the road feature in third image data captured by the third camera, and (iii) determining, via comparing the determined movements of the road feature, a rotational and translational offset of the third camera at the vehicle relative to the second camera at the vehicle.

17. The method of claim 14, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a horizontal shift of the road feature while the vehicle is moving along the road.

18. The method of claim 14, wherein determining the rotational and translational offset of the second camera at the vehicle relative to the first camera at the vehicle comprises determining, via comparing the determined movements of the road feature, a vertical shift of the road feature while the vehicle is moving along the road.