Image processing apparatus and method
First Claim
1. An image processor, comprising:
- means for providing control information;
an image correcting unit operable to align object images acquired by imaging an object by at least two cameras from different points of view based on said control information to form at least two corrected object images, the corrected object images each including a plurality of horizontal lines and a plurality of pixels on each of the horizontal lines, and the difference in brightness between the corrected object images being minimized through the use of projective transformation matrices corresponding to the object images, the correcting unit being operable to match images from the at least two cameras with a reference image through a projective transformation which is based on at least a chromatic aberration, a distortion, and an optical-axial misalignment of the lens for each of the cameras; and
a matching unit operable to determine a correspondence between at least one pixel position on a horizontal line in one of the corrected object images and at least one pixel position on the same horizontal line in another of the corrected object images by determining a similarity in a brightness component and a color component at the pixel positions and a parallax between the corrected object images,in which said control information includes information indicative of a positional relationship of each camera to the object, the information indicative of a positional relationship being generated by using the object images to acquire the line of vision between the object and a display associated with the cameras and generating the information indicative of a positional relationship based on the acquired line of vision, andin which, to determine the correspondence, the matching unit determines whether priority is to be given to the similarity or to the parallax, and the matching unit matches a plurality of pixel positions on the horizontal line in the one of the corrected object images with the same pixel positions on the same horizontal line in the another of the corrected object images when priority is given to the similarity, and the matching unit matches a plurality of pixel positions on the horizontal line in one of the corrected object images with one pixel position on the same horizontal line in the another of the corrected object images when priority is given to the parallax,the control information including, for each camera, at least one of an indication of imaging direction and an indication of angle of view, and the correction unit using the at least one of the indication of imaging direction and the indication of angle of view for performing a geometric normalization of the corrected object images.
1 Assignment
0 Petitions
Accused Products
Abstract
Directions normal to images of an object captured by at least two cameras from different points of view are aligned with each other to form corrected object images, and a correspondence is determined between at least one pixel position on a horizontal line in one of the corrected object images and at least one pixel position on the same horizontal line in another of the corrected object images. The correspondence is determined by determining a similarity in brightness and color components at the pixel positions and a parallax between the corrected object images A robust, accurate image matching can be done by making dynamic matching between all pixels on a scan line in the images captured by the cameras.
-
Citations
14 Claims
-
1. An image processor, comprising:
-
means for providing control information; an image correcting unit operable to align object images acquired by imaging an object by at least two cameras from different points of view based on said control information to form at least two corrected object images, the corrected object images each including a plurality of horizontal lines and a plurality of pixels on each of the horizontal lines, and the difference in brightness between the corrected object images being minimized through the use of projective transformation matrices corresponding to the object images, the correcting unit being operable to match images from the at least two cameras with a reference image through a projective transformation which is based on at least a chromatic aberration, a distortion, and an optical-axial misalignment of the lens for each of the cameras; and a matching unit operable to determine a correspondence between at least one pixel position on a horizontal line in one of the corrected object images and at least one pixel position on the same horizontal line in another of the corrected object images by determining a similarity in a brightness component and a color component at the pixel positions and a parallax between the corrected object images, in which said control information includes information indicative of a positional relationship of each camera to the object, the information indicative of a positional relationship being generated by using the object images to acquire the line of vision between the object and a display associated with the cameras and generating the information indicative of a positional relationship based on the acquired line of vision, and in which, to determine the correspondence, the matching unit determines whether priority is to be given to the similarity or to the parallax, and the matching unit matches a plurality of pixel positions on the horizontal line in the one of the corrected object images with the same pixel positions on the same horizontal line in the another of the corrected object images when priority is given to the similarity, and the matching unit matches a plurality of pixel positions on the horizontal line in one of the corrected object images with one pixel position on the same horizontal line in the another of the corrected object images when priority is given to the parallax, the control information including, for each camera, at least one of an indication of imaging direction and an indication of angle of view, and the correction unit using the at least one of the indication of imaging direction and the indication of angle of view for performing a geometric normalization of the corrected object images. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. An image processing method of using a processor to perform steps comprising:
-
providing control information; aligning object images acquired by imaging an object by at least two cameras from different points of view based on said control information to form at least two corrected object images, the corrected object images each including a plurality of horizontal lines and a plurality of pixels on each of the horizontal lines, and the difference in brightness between the corrected object images being minimized through the use of projective transformation matrices corresponding to the object images, a correcting unit being used to match images from the at least two cameras with a reference image through a projective transformation which is based on at least a chromatic aberration, a distortion, and an optical-axial misalignment of the lens for each of the cameras; and using a matching unit for determining a correspondence between at least one pixel position on a horizontal line in one of the corrected object images and at least one pixel position on the same horizontal line in another of the corrected object images by determining a similarity in a brightness component and a color component at the pixel positions and a parallax between the corrected object images, in which said control information includes information indicative of a positional relationship of each camera to the object, the information indicative of a positional relationship being generated by using the object images to acquire the line of vision between the object and a display associated with the cameras and generating the information indicative of a positional relationship based on the acquired line of vision, in which the step of determining the correspondence includes determining whether priority is to be given to the similarity or to the parallax, and in which a plurality of pixel positions on the horizontal line in the one of the corrected object images is matched with the same pixel positions on the same horizontal line in the another of the corrected object images when priority is given to the similarity, and a plurality of pixel positions on the horizontal line in the one of the corrected object images is matched with one pixel position on the same horizontal line in the another of the corrected object images when priority is given to the parallax, the control information including, for each camera, at least one of an indication of imaging direction and an indication of angle of view, and the method further comprising using the at least one of the indication of imaging direction and the indication of angle of view for performing a geometric normalization of the corrected object images. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
Specification