System and method of establishing a multi-camera image using pixel remapping
First Claim
1. A method of establishing a composite image for displaying in a vehicle, comprising:
- a) providing a first camera and a second camera, each camera having a field of view;
b) positioning the cameras at the vehicle so that the fields of view of the cameras overlap partially, wherein the cameras together have a combined field of view, and wherein the first camera comprises a rear backup camera having a field of view rearward of the vehicle;
c) recording preliminary digital images from the cameras, each preliminary digital image being made up of a plurality of pixels;
d) sequentially performing at least three functions on the preliminary digital images, wherein the at least three functions comprise (i) dewarping the preliminary digital images to generate intermediate dewarped images and storing the pixel remapping used to generate the intermediate dewarped images in a first intermediate remapping table, (ii) viewpoint adjustment of the intermediate dewarped images to generate dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted images in a second intermediate remapping table, and (iii) offset correction of the dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted and offset-corrected images in a third intermediate remapping table;
e) generating a combined remapping table by combining remapping information of the first, second and third intermediate remapping tables;
f) generating a final composite digital image that corresponds to a selected digital representation of the combined field of view of the cameras, at least in part by remapping selected pixels from each of the preliminary digital images of the first and second cameras into selected positions of the final composite digital image in accordance with the combined remapping table;
g) displaying, at a video display screen during a backup maneuver of the vehicle, intermediate dewarped images derived from applying the pixel remapping of the first intermediate remapping table on the preliminary digital images of the first camera;
h) displaying, at the video display screen, the final composite digital images when parking the vehicle; and
i) wherein the video display screen is operable to display a split view of images.
1 Assignment
0 Petitions
Accused Products
Abstract
A camera or vision system for establishing a composite image for displaying in a vehicle includes a first camera and a second camera and a controller. Each camera has a respective field of view that overlaps partially with the respective field of view of the other camera. Each camera has a respective imager for generating a respective preliminary digital image. The cameras together have a combined field of view. The controller is programmed to generate a final composite digital image that corresponds to a selected digital representation of the combined field of view of the cameras by using a remapping table to remap selected pixels from each of the preliminary digital images into selected positions of the final composite digital image. A plurality of methods for establishing a composite image are also provided.
-
Citations
23 Claims
-
1. A method of establishing a composite image for displaying in a vehicle, comprising:
-
a) providing a first camera and a second camera, each camera having a field of view; b) positioning the cameras at the vehicle so that the fields of view of the cameras overlap partially, wherein the cameras together have a combined field of view, and wherein the first camera comprises a rear backup camera having a field of view rearward of the vehicle; c) recording preliminary digital images from the cameras, each preliminary digital image being made up of a plurality of pixels; d) sequentially performing at least three functions on the preliminary digital images, wherein the at least three functions comprise (i) dewarping the preliminary digital images to generate intermediate dewarped images and storing the pixel remapping used to generate the intermediate dewarped images in a first intermediate remapping table, (ii) viewpoint adjustment of the intermediate dewarped images to generate dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted images in a second intermediate remapping table, and (iii) offset correction of the dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted and offset-corrected images in a third intermediate remapping table; e) generating a combined remapping table by combining remapping information of the first, second and third intermediate remapping tables; f) generating a final composite digital image that corresponds to a selected digital representation of the combined field of view of the cameras, at least in part by remapping selected pixels from each of the preliminary digital images of the first and second cameras into selected positions of the final composite digital image in accordance with the combined remapping table; g) displaying, at a video display screen during a backup maneuver of the vehicle, intermediate dewarped images derived from applying the pixel remapping of the first intermediate remapping table on the preliminary digital images of the first camera; h) displaying, at the video display screen, the final composite digital images when parking the vehicle; and i) wherein the video display screen is operable to display a split view of images. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 13, 14, 15, 22)
-
-
9. A method of establishing a composite image for displaying in a vehicle, comprising:
-
a) providing a first camera and a second camera, a third camera and a fourth camera, each camera having a field of view, wherein the cameras together have a combined field of view that is a 360 degree field of view around the vehicle; b) positioning the cameras at the vehicle so that the field of view of each camera overlaps partially with the field of view of two of the other cameras, and wherein the field of view of the first camera overlaps partially with the field of view of the second camera and the field of view of the third camera, and wherein the first camera comprises a rear backup camera having a field of view rearward of the vehicle; c) recording preliminary digital images from the cameras, each preliminary digital image being made up of a plurality of pixels; d) sequentially performing at least three functions on the preliminary digital images, wherein the at least three functions comprise (i) dewarping the preliminary digital images to generate intermediate dewarped images and storing the pixel remapping used to generate the intermediate dewarped images in a first intermediate remapping table, (ii) viewpoint adjustment of the intermediate dewarped images to generate dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted images in a second intermediate remapping table, and (iii) offset correction of the dewarped and viewpoint-adjusted images and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted and offset-corrected images in a third intermediate remapping table; e) generating a combined remapping table by combining remapping information of the first, second and third intermediate remapping tables; f) generating a final composite digital image that corresponds to a selected digital representation of the combined field of view of the cameras by remapping selected pixels from each of the preliminary digital images into selected positions of the final composite digital image in accordance with the combined remapping table, wherein the preliminary digital images each have associated therewith a preliminary apparent camera viewpoint and the final composite digital image has associated therewith a final apparent camera viewpoint, and wherein the selected pixels from the preliminary digital images are selected so that the final apparent camera viewpoint associated with the final composite digital image is higher than the preliminary apparent camera viewpoints associated with the preliminary digital images, and wherein the selected pixels from the preliminary digital images are selected so that any misalignment between the overlapping portions of the preliminary digital images is substantially eliminated, and wherein the selected pixels from the preliminary digital images are selected so that the final composite digital image is dewarped as compared to each of the preliminary digital images; g) displaying, at a video display screen during a backup maneuver of the vehicle, intermediate dewarped images derived from applying the pixel remapping of the first intermediate remapping table on the preliminary digital images of the first camera; h) displaying, at the video display screen, the final composite digital images when parking the vehicle; and i) wherein the video display screen is operable to display a split view of images. - View Dependent Claims (16, 23)
-
-
10. A method of generating a remapping table for use in mapping pixels from a plurality of preliminary digital images into a final composite image, comprising:
-
a) providing a first camera and a second camera at a vehicle such that each camera has a field of view that overlaps partially with the field of view of the other camera, each camera having an imager for generating one of the preliminary digital image, wherein the cameras together have a combined field of view, and wherein the first camera comprises a rear backup camera having a field of view rearward of the vehicle; b) detecting a target feature along the path of the vehicle while the vehicle is driven, using a controller at the vehicle; c) providing a first preliminary digital image from the first camera, wherein the first preliminary digital image contains a first representation of the target feature at a first point time; d) determining the position of the first representation of the target feature in the first preliminary digital image; e) providing a second preliminary digital image from the second camera, wherein the second preliminary digital image contains a second representation of the target feature at a second point time; f) determining the position of the second representation of the target feature in the second preliminary digital image; g) comparing the positions of the first and second representations of the target feature; h) generating at least one value for the remapping table based on the result of the comparison in step g); i) sequentially performing at least three functions on the at least one preliminary digital image, wherein the at least three functions comprise (i) dewarping the at least one preliminary digital image to generate at least one intermediate dewarped image and storing the pixel remapping used to generate the intermediate dewarped images in a first intermediate remapping table, (ii) viewpoint adjustment of the at least one intermediate dewarped image to generate at least one dewarped and viewpoint-adjusted image and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted images in a second intermediate remapping table, and (iii) offset correction of the at least one dewarped and viewpoint-adjusted image and storing the pixel remapping used to generate the dewarped and viewpoint-adjusted and offset-corrected images in a third intermediate remapping table; j) generating a combined remapping table by combining remapping information of the first, second and third intermediate remapping tables; k) generating at least one final composite digital image that corresponds to a selected digital representation of the combined field of view of the cameras by remapping selected pixels from the at least one preliminary digital image into selected positions of the at least one final composite digital image in accordance with the combined remapping table; l) displaying, at a video display screen during a backup maneuver of the vehicle, intermediate dewarped images derived from applying the pixel remapping of the first intermediate remapping table on the preliminary digital images of the first camera; m) displaying, at the video display screen, the final composite digital images when parking the vehicle; and n) wherein the video display screen is operable to display a split view of images. - View Dependent Claims (11, 12, 17, 18, 19, 20, 21)
-
Specification