IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20200137379A1
• Filed: 06/13/2018
• Published: 04/30/2020
• Est. Priority Date: 06/27/2017
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: TSUNASHIMA, NOBUHIRO
• Granted Patent: US 11,044,463 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/302 for viewing without the aid of special glasses, i.e. using autostereoscopic displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388

Abstract

The present technology relates to an image processing device, an image processing method, and a program that enable more accurate rectification of a stereo image captured by a plurality of wide-angle cameras. An image processing device according to one aspect of the present technology is a device that acquires a stereo image captured by a plurality of cameras each including a wide-angle lens, divides viewing angles of the cameras with reference to optical axes corrected to be parallel to each other, generates a plurality of base images in each of which a range of each divided viewing angle is reflected and a plurality of reference images on a basis of wide-angle images constituting the stereo image, applies projective transformation to the reference images, and calculates a distance to a predetermined object on a basis of corresponding image pairs of the plurality of base images and the plurality of reference images after projective transformation. The present technology can be applied to an in-vehicle device.

Claims

1. An image processing device comprising:

an acquisition unit configured to acquire a stereo image captured by a plurality of cameras each including a wide-angle lens;

a generation unit configured to divide viewing angles of the cameras with reference to optical axes corrected to be parallel to each other and generate a plurality of base images in each of which a range of each divided viewing angle is reflected and a plurality of reference images on a basis of wide-angle images constituting the stereo image;

a projective transformation unit configured to apply projective transformation to the reference images; and

a distance calculation unit configured to calculate a distance to a predetermined object on a basis of corresponding image pairs of the plurality of base images and the plurality of reference images after projective transformation.

2. The image processing device according to claim 1, further comprising:

a projection unit configured to project a first wide-angle image and a second wide-angle image constituting the stereo image onto virtual spherical surfaces including the viewing angles of the cameras, respectively, wherein

the generation unit

reprojects the first wide-angle image projected onto the virtual spherical surface onto a plurality of planes on the virtual spherical surface to generate the plurality of base images, and

reprojects the second wide-angle image projected onto the virtual spherical surface onto a plurality of planes on the virtual spherical surface to generate the plurality of reference images.

3. The image processing device according to claim 1, further comprising:

a correction unit configured to correct the optical axis on a basis of the stereo image in which a known object is reflected; and

a storage unit configured to store information regarding the optical axis after correction.

4. The image processing device according to claim 3, further comprising:

a parameter generation unit configured to generate a parameter to be used for projective transformation on a basis of corresponding points of the base image and the reference image constituting the image pair, wherein

the storage unit further stores the parameter.

5. The image processing device according to claim 4, wherein

the correction unit sets the optical axis after correction on a basis of information stored in the storage unit, and

the projective transformation unit performs the projective transformation of the reference image on a basis of the parameter stored in the storage unit.

6. The image processing device according to claim 3, wherein

the correction unit repeatedly performs the correction of the optical axis until a correction error becomes equal to or less than a threshold.

7. The image processing device according to claim 1, wherein

the acquisition unit acquires wide-angle images captured by two of the cameras as the stereo image.

8. The image processing device according to claim 1, further comprising:

the plurality of cameras.

9. An image processing method comprising the steps of:

acquiring a stereo image captured by a plurality of cameras each including a wide-angle lens;

dividing viewing angles of the cameras with reference to optical axes corrected to be parallel to each other;

generating a plurality of base images in each of which a range of each divided viewing angle is reflected and a plurality of reference images on a basis of wide-angle images constituting the stereo image;

applying projective transformation to the reference images; and

calculating a distance to a predetermined object on a basis of corresponding image pairs of the plurality of base images and the plurality of reference images after projective transformation.

10. A program for causing a computer to execute processing comprising the steps of:

acquiring a stereo image captured by a plurality of cameras each including a wide-angle lens;

dividing viewing angles of the cameras with reference to optical axes corrected to be parallel to each other;

generating a plurality of base images in each of which a range of each divided viewing angle is reflected and a plurality of reference images on a basis of wide-angle images constituting the stereo image;

applying projective transformation to the reference images; and

calculating a distance to a predetermined object on a basis of corresponding image pairs of the plurality of base images and the plurality of reference images after projective transformation.