Method of matching stereo images and method of measuring disparity between these image

• US 5,867,591 A
• Filed: 04/09/1996
• Issued: 02/02/1999
• Est. Priority Date: 04/21/1995
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Corporation)
• Inventors: Onda, Katsumasa
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): MEHTA, BHAVESH M
• Time in patent office: 1029 Days
• Field of Search: 382/154, 382/153, 382/168, 382/171, 382/173, 382/209, 382/255, 382/276-280, 382/285, 382/293-294, 382/106, 356/2, 356/12, 348/42, 348/43, 348/47, 345/419, 345/425, 345/427, 345/139
• US Class Current: 382/154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06K 2209/40 Acquisition of 3D measurements of objects
• CPC Class Codes: G06K 9/20 Image acquisition
• CPC Class Codes: G06K 9/32 Aligning or centering of the image pick-up or image-field
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

In the image pickup phase (A), right and left images are taken in through two image-pickup devices (S101, S102). Then, in the next feature extraction phase (B), right and left images are respectively subjected to feature extraction (S103, S104). Thereafter, in the succeeding matching phase (C), the extracted features of right and left images are compared to check how they match with each other (step S105). More specifically, in the matching phase (C), a one-dimensional window is set, this one-dimensional window is shifted along the left image in accordance with a predetermined scanning rule so as to successively set overlapped one-dimensional windows, and a matching operation is performed by comparing the image features within one window and corresponding image features on the right image. Subsequently, in the disparity determination phase (D), the left image is dissected or divided into plural blocks each having a predetermined size, a histogram in each block is created from disparities obtained by the matching operation based on one-dimensional windows involving pixels of a concerned block, and a specific disparity just corresponding to the peak of thus obtained histogram is identified as a valid disparity representing the concerned block (S106).

Claims

1. A method of detecting a disparity between stereo images, comprising the steps of:

dividing each of first and second images IL and IR into a plurality of blocks each having a size of M×

L pixels;

matching ternary-valued frequency component images of said images IL and IR;

comparing pixels in a micro region defined by a one-dimensional window set on said first image IL with pixels in a designated micro region on said second image IR;

evaluating a similarity between two micro regions using the following equation;

space="preserve" listing-type="equation">Ε

all=Σ

β

k(PN)k+Σ

γ

k(ZN)k

where PN represents a total number of pixels having an evaluation result "P" while ZN represents a total number of pixels having an evaluation result "Z", and β

k and γ

k represent weighting factors;

searching a first region having a most highest similarity and a second region having a second highest similarity in a concerned block;

specifying a first candidate disparity as a disparity corresponding to said first region, and a second candidate disparity as a disparity corresponding to said second region;

creating a histogram based on said first and second candidate disparities; and

determining a valid disparity of said concerned block as a disparity corresponding to a peak position of said histogram.

2. The method defined by claim 1, wherein said first image IL is designated as a reference image, a one-dimensional window capable of encompassing N pixels therein is set on the ternary-valued frequency component image of said first image IL, and a matching region having the same ternary-value pattern as said N pixels in said one-dimensional window is searched from the ternary-valued frequency component image of said second image IR.

3. The method defined by claim 1, wherein one of said first and second images IL and IR is designated as a reference image, a plurality of one-dimensional windows are set on the entire surface of said ternary-valued frequency component image of said reference image through a scanning operation along an epipolar line, so that said one-dimensional windows are successively overlapped at the same intervals of N/2 when each of said one-dimensional windows has a size equivalent to N pixels, and a matching operation is carried out with respect to each of said one-dimensional windows.

4. The method defined by claim 1, wherein said valid disparity is calculated as a sub-pixel level parity corresponding to an intersecting point of a first straight line crossing two points (di-1, hi-1), (di, hi) and a second straight line crossing a point (di+1, hi+1) with a gradient symmetrical with said first straight line, where di-1, di, di+1 represent disparities near the peak position of said histogram and hi-1, hi, hi+represent the number of occurrences of said disparities di-1, di, di+1 respectively.