STEREO IMAGE PROCESSING DEVICE AND STEREO IMAGE PROCESSING METHOD

• US 20150249814A1
• Filed: 07/04/2013
• Published: 09/03/2015
• Est. Priority Date: 09/27/2012
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Nanri, Takuya, Kurokawa, Hisashi, Ishihara, Takeshi, Nishimura, Hirofumi
• Granted Patent: US 10,009,594 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/026 by measuring distance between sensor and object G01B11/0608 takes precedence
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Device (100) comprises: a stereo image acquisition unit (101) for acquiring a standard image and a reference image for a stereo image; an image segmenting unit (102) for acquiring standard pixel data and reference pixel data from the stereo image; a phase correction unit (103) for carrying out for each of the pixel data a phase correction process making the values of the standard pixel data more symmetrical; a windowing unit (104) for applying a windowing function to each of the pixel data that have undergone phase correction; a phase correlation unit 105 for calculating a phase correlation value between the two pixel data that have undergone windowing; and a peak position detection unit (106) for calculating, on the basis of the phase correlation value, the parallax of a corresponding point of the reference image with respect to a standard point on the standard image.

Claims

1. A stereo image processing apparatus comprising:

a stereo image acquiring section that acquires a target image and a reference image of a stereo image;

an image clipping section that clips partial images parallel to a baseline length direction of the stereo image from the target image and the reference image respectively and acquires one-dimensional pixel data sequences each indicating a distribution of pixel values of a corresponding one of the partial images in the baseline length direction;

a phase correction section that performs a phase correction process on each of target pixel data and reference pixel data, the phase correction process being a process which makes values of the target pixel data more symmetric when applied to the target pixel data that is the one-dimensional pixel data sequence acquired from the target image, the reference pixel data being the one-dimensional pixel data sequence acquired from the reference image;

a windowing section that performs windowing on phase-corrected target pixel data obtained by performing the phase correction process on the target pixel data and phase-corrected reference pixel data obtained by performing the phase correction process on the reference pixel data respectively;

a phase correlation section that calculates a phase correlation value indicating a degree of correlation between windowed target pixel data obtained by performing the windowing on the phase-corrected target pixel data and windowed reference pixel data obtained by performing the windowing on the phase-corrected reference pixel data; and

a peak position detecting section that calculates a disparity of a corresponding point in the reference image corresponding to a target point in the target image based on the phase correlation value.

2. The stereo image processing apparatus according to claim 1, wherein

the phase correction section comprises:

a target pixel data phase zeroing section that performs a phase zeroing process of zeroing a phase of each frequency component on the target pixel data and thereby calculates the phase-corrected target pixel data;

a phase correction filter calculating section that calculates a phase correction filter that makes values of the target pixel data symmetric based on the target pixel data and the phase-corrected target pixel data when applied to the target pixel data; and

a reference pixel data phase correction section that calculates the phase-corrected reference pixel data by applying the phase correction filter to the reference pixel data.

3. The stereo image processing apparatus according to claim 2, wherein

the phase zeroing process comprises a process in which one-dimensional Fourier transform, removal of an imaginary number component and one-dimensional inverse Fourier transform are performed on the target pixel data in the order named, and

the phase correction filter comprises a filter calculated by dividing a value obtained by performing one-dimensional Fourier transform on the phase-corrected target pixel data, by a value obtained by performing one-dimensional Fourier transform on the target pixel data.

4. The stereo image processing apparatus according to claim 1, wherein

the phase correction section comprises:

a target pixel data phase shift section that applies a phase shift filter which shifts phases of some or all frequency components by 90 degrees with respect to the target pixel data and that calculates phase-shifted target pixel data;

a symmetry determination value calculating section that calculates symmetry determination values indicating symmetry of values for the phase-shifted target pixel data and the reference pixel data respectively;

a symmetry determination value comparison section that compares the symmetry determination value calculated from the phase-shifted target pixel data with the symmetry determination value calculated from the target pixel data and that determines whether or not the symmetry of the phase-shifted target pixel data is higher than the symmetry of the target pixel data; and

a reference pixel data phase shift section that applies the phase shift filter to the reference pixel data and that calculates phase-shifted reference pixel data, and

the windowing section handles, when the symmetry of the phase-shifted target pixel data is higher than the symmetry of the target pixel data, the phase-shifted target pixel data and the phase-shifted reference pixel data as the phase-corrected target pixel data and the phase-corrected reference pixel data respectively, and handles, when the symmetry of the phase-shifted target pixel data is not higher than the symmetry of the target pixel data, the target pixel data and the reference pixel data as the phase-corrected target pixel data and the phase-corrected reference pixel data respectively.

5. The stereo image processing apparatus according to claim 4, wherein the phase shift filter is a differential filter.

6. The stereo image processing apparatus according to claim 1, wherein the phase correlation section calculates an inverted phase filtering coefficient by inverting the windowed target pixel data in an array direction, filters the windowed reference pixel data using the inverted phase filtering coefficient and thereby calculates the phase correlation value.

7. The stereo image processing apparatus according to claim 1, further comprising an image matching section that acquires the corresponding point on a pixel level, wherein

the image clipping section determines a clipping position of the partial image so that the target point and the corresponding point are included in the partial image, and

the peak position detecting section calculates a disparity of the corresponding point on a subpixel level corresponding to a target point in the target image based on the phase correlation value.

8. A stereo image processing method comprising:

acquiring a target image and a reference image of a stereo image;

clipping partial images parallel to a baseline length direction of the stereo image from the target image and the reference image respectively and acquiring one-dimensional pixel data sequences each indicating a distribution of pixel values of a corresponding one of the partial images in the baseline length direction;

performing a phase correction process on each of target pixel data and reference pixel data, the phase correction process being a process which makes values of the target pixel data more symmetric when applied to the target pixel data that is the one-dimensional pixel data sequence acquired from the target image, the reference pixel data being the one-dimensional pixel data sequence acquired from the reference image;

performing windowing on phase-corrected target pixel data obtained by performing the phase correction process on the target pixel data and phase-corrected reference pixel data obtained by performing the phase correction process on the reference pixel data respectively;

calculating a phase correlation value indicating a degree of correlation between windowed target pixel data obtained by performing the windowing on the phase-corrected target pixel data and windowed reference pixel data obtained by performing the windowing on the phase-corrected reference pixel data; and

calculating a disparity of a corresponding point in the reference image corresponding to a target point in the target image based on the phase correlation value.