Image processor and image processing method for a vehicle

• US 20060197761A1
• Filed: 03/02/2006
• Published: 09/07/2006
• Est. Priority Date: 03/03/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Akatsuka, Takeshi, Oizumi, Ken, Suzuki, Masayasu
• Granted Patent: US 7,782,374 B2
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image processor is disclosed for a vehicle having a plurality of cameras and a monitor that displays a panoramic image derived from multiple images generated by the plurality of cameras. The processor includes input buffers for storing camera image data captured through respective camera modules and output buffer for storing a panoramic image. The camera image data for generating the panoramic image are retrieved from input buffers. The camera image data are stored in an output buffer using an reconfigured address conversion table. The reconfigured address conversion table is an address conversion table which is obtained by reconfiguring an address conversion table when camera modules are assumed to be free of vehicle installation errors. The reconfiguration is performed on the basis of parameters which are used to correct vehicle installation errors of camera modules.

Claims

1. An image processor for a vehicle having a plurality of cameras and a monitor that displays a panoramic image derived from multiple images generated by the plurality of cameras, the image processor comprising:

input buffers adapted to receive and store camera image data generated by the plurality cameras;

an output buffer adapted to store panoramic image data for display on the monitor;

a reconfigured address conversion table that describes the relationship between the camera image data and the panoramic image data, which reconfigured address conversion table is prepared by calibrating a non-reconfigured address conversion table to correct for installation errors in the plurality of cameras; and

a controller adapted to read the camera image data stored on the input buffers, translate the camera image data into the panoramic image data using the reconfigured address conversion table, and store the panoramic image data on the output buffer.

2. The image processor of claim 1, wherein:

the reconfigured address conversion table includes one or more numbers each assigned to a particular camera to designate input data contained in a specified input buffer as having been generated by that particular camera.

3. The image processor of claim 1, wherein the input buffers are part of a single memory unit.

4. The image processor of claim 1, wherein:

the reconfigured address conversion table includes input address information that associates each of a plurality of input addresses in the input buffers with a corresponding output memory address in the output buffer to form at least a portion of the panoramic image; and

the controller is further adapted to read the camera image data located at each of the plurality of input addresses and to store the camera image data so read at the corresponding output memory address in accordance with the input address information.

5. The image processor of claim 4, further comprising a table reconfiguration module that corrects installation errors in the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table and storing the shifted input address information in the reconfigured address conversion table.

6. The image processor of claim 5, wherein the table reconfiguration module corrects a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

7. The image processor of claim 5, wherein the table reconfiguration module corrects a pitch error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

8. The image processor of claim 5, wherein the table reconfiguration module corrects a roll error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

9. The image processor of claim 5, wherein the table reconfiguration module corrects a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras, and corrects a pitch error in the installation of the one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system.

10. The image processor of claim 5, wherein the table reconfiguration module first corrects a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras, and corrects a pitch error in the installation of the one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system;

and then corrects a roll error in the installation of the one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in the image coordinate system.

11. An image processor for a vehicle having a plurality of cameras and a monitor that displays a panoramic image generated from multiple images generated by the plurality of cameras, the image processor comprising:

input buffer means for storing camera image data captured by the cameras;

output buffer means for storing panoramic image data generated from the camera image data retrieved from the input buffer means and for outputting the panoramic image data to the monitor;

reconfigured address conversion table means which associates a plurality of input addresses in the input buffer means with a corresponding output memory addresses in the output buffer means to form at least a portion of the panoramic image data; and

image processing means for retrieving the camera image data from the input buffer means and storing the camera image data in the output buffer means according to the reconfiguration address conversion table means to generate the panoramic image data;

wherein the reconfigured address conversion table means is generated from a non-reconfigured address conversion table, used to generate at least a portion of the panoramic image data when the cameras are assumed to be free of installation errors, that is calibrated to correct installation errors of the cameras.

12. The image processor of claim 11, wherein:

the reconfigured address conversion table means includes one or more camera numbers each assigned to a particular camera to designate input data contained in a specified input buffer as having been generated by that particular camera; and

the reconfigured address conversion table means describes the correspondence between input address information for designating input addresses needed for retrieving the camera image data for generating the panoramic image data from the input buffers, and output address information for designating output addresses in the output buffer means that contain the camera image data retrieved from the input addresses; and

the image processing means retrieves the camera image data from the input buffers according to the camera numbers and the input address information designated by the reconfigured address conversion tables and stores the camera image data in output addresses in the output buffer according to the output address information designated by the reconfigured address conversion tables means.

13. The vehicle image processor of claim 11, further comprising:

table reconfiguration means for shifting the input addresses of the non-reconfigured address conversion table to correct installation errors of the cameras on the vehicle in order to generate the reconfigured address conversion table means.

14. The vehicle image processor of claim 13, wherein the table reconfiguration means corrects for a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

15. The vehicle image processor of claim 13, wherein the table reconfiguration means corrects a pitch error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

16. The image processor of claim 13, wherein the table reconfiguration means corrects a roll error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

17. The vehicle image processor of claim 13, wherein the table reconfiguration means corrects a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras, and corrects a pitch error in the installation of the one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system.

18. The vehicle image processor of claim 13, wherein the table reconfiguration means first corrects a yaw error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras, and corrects a pitch error in the installation of the one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system;

and then after the yaw error and pitch error are corrected, corrects a roll error in the installation of one of the plurality of cameras by shifting at least one of the plurality of input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in an image coordinate system represented by the camera image data generated by the one of the plurality of cameras.

19. An image processing method for use in a vehicle having a plurality of cameras and a monitor that displays a panoramic image derived from multiple images generated by the plurality of cameras, comprising

storing camera image data captured by the cameras on an input buffer;

providing an output buffer on which to store panoramic image data generated from the camera image data retrieved from the input buffer and to output the panoramic image to the monitor;

providing a non-reconfigured address conversion table that associates a plurality of input addresses in the input buffer with corresponding output memory addresses in the output buffer to form at least a portion of the panoramic image when the cameras are assumed to be free of installation errors;

determining the existence of an installation error in at least one of the plurality of cameras and the correction parameter to be applied to the non-reconfigured address conversion table to correct the installation error;

reconfiguring the non-reconfigured address conversion table by shifting the input addresses in accordance with the correction parameter; and

retrieving the camera image data from the input buffer and storing the camera image data in the output buffer according to the reconfigured address conversion table to generate the panoramic image.

20. The image processing method of claim 19, further comprising:

correcting for a yaw error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras.

21. The image processing method of claim 19, further comprising:

correcting for a pitch error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in an image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras.

22. The image processing method of claim 19, further comprising:

correcting for a roll error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in an image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras.

23. The image processing method of claim 19, further comprising:

correcting for a yaw error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras; and

correcting for a pitch error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras.

24. The image processing method of claim 19, further comprising:

first correcting for a yaw error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a horizontal direction in an image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras; and

correcting for a pitch error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a vertical direction in the image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras;

then after the yaw and pitch errors have been corrected, correcting for a roll error in the installation of the at least one of the plurality of cameras by shifting the input addresses of the non-reconfigured address conversion table in a direction corresponding to a rotational direction in the image coordinate system represented by the camera image data generated by the at least one of the plurality of cameras.

25. A method for calibrating an vehicle image processor that incorporates camera image data generated by multiple vehicle-mounted cameras and stored on an input buffer into a panoramic image stored on an output buffer for display on a monitor, comprising:

generating a non-reconfigured address conversion table to associate at least one selected input address in the input buffer with an output address in the output buffer to create at least a portion of a panoramic image in the output buffer;

detecting errors in the installation of at least one of the multiple vehicle-mounted cameras;

creating a reconfigured address conversion table by changing the non-reconfigured address table so that it associates a different input addresses in the input buffer with the target output address in the output buffer, the different input address being offset from the selected input address in a direction opposing the detected error in the installation of the at least one vehicle-mounted camera; and

loading the reconfigured address conversion table into a memory accessible by the vehicle image processor in a manner that permits the vehicle image processor to retrieve camera image data stored at the different input address and store the retrieved camera image data in the target output address of the output buffer.