System and methods to efficiently measure stereoscopic disparity and to generate related mismatch images

• US 9,336,596 B2
• Filed: 01/11/2011
• Issued: 05/10/2016
• Est. Priority Date: 12/28/2010
• Status: Expired due to Fees

• Current Assignee: Project Giants, LLC
• Original Assignee: Tektronix Incorporated (Fortive Corp.)
• Inventors: Ferguson, Kevin M.
• Primary Examiner(s): Rao, Andy
• Application Number:
  US13/004,124
• Publication Number:
  US 20120162364A1
• Time in Patent Office: 1,946 Days
• Field of Search: 348 42- 60
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Embodiments of the invention include a system and methods for measuring disparity and mismatch of stereoscopic images of three-dimensional video, which may indicate a level of discomfort on the part of the viewer. Measuring these parameters may also be used to give an indication that certain processes have or have not adversely effected perceived depth (i.e. through conversion from disparity to depth given a display size and viewing distance), verify camera setup or 2-dimension to 3-dimension synthesizing processes. Generating the indications and other data about the stereoscopic images uses techniques that allow processing with much less computing resources than was possible in previous systems.

Claims

1. A method, comprising:

accepting a left-eye image and a right-eye image of a stereoscopic image;

creating a set of differential data from differences between at least a portion of the left image and at least a portion of the right image by performing a set of pixelwise comparisons between the portion of the left image to the portion of the right image in which, for each pixelwise comparison, the portion of the left image and the portion of the right image are incrementally shifted relative to each other;

generating a two-dimensional mismatch image in which each pixel of a line of the mismatch image corresponds to one of the set of pixelwise comparisons; and

assigning a color to each pixel of the mismatch image, in which the color is indicative of a level of match of the set of differential data.

2. The method of claim 1 in which the left-eye image and the right-eye image have a height and width, and in which the set of differential data is created from a vertically downsampled version of the left-eye image and the right-eye image.

3. The method of claim 2 in which each group of data in the set of differential data corresponds to a vertically downsampled image line of the left-eye image and right-eye image.

4. The method of claim 3 in which each line of the mismatch image is vertically expanded to fill a region of the mismatch image that corresponds to the region of the stereoscopic image from which the line was created.

5. A method of generating differential data from a stereoscopic image having a left-eye image and a right-eye image, the method comprising:

vertically downsampling at least one of the left-eye image and the right-eye image into a set of downsampled image lines;

creating an unsharp mask for a downsampled image line of the set of downsampled image lines by filtering the downsampled image line with an IIR low pass filter to create a filtered downsampled image line and removing the filtered downsampled image line from the downsampled image line; and

generating a set of difference data for the downsampled image line by comparing the downsampled image line to its own unsharp mask and storing the comparison, the set of difference data comprising absolute value differences.

6. The method of claim 5, in which the at least one of the left-eye image and right-eye image is a left-eye image, the method further comprising:

vertically downsampling the right-eye image into a second set of downsampled image lines;

creating an unsharp mask for a downsampled image line of the second set of downsampled image lines by filtering the downsampled image line of the second set of downsampled image lines with the IIR low pass filter to create a second filtered downsampled image line and removing the second filtered downsampled image line from the downsampled image line of the second set of downsampled image lines;

generating a set of difference data for the downsampled image line of the second set of downsampled image lines by comparing the downsampled image line of the second set of downsampled image lines to its own unsharp mask and storing the comparison; and

performing a plurality of comparisons of the stored sets of comparison data from the left-eye image and the right-eye image to one another, in which, for each comparison, the left-eye image and the right image are incrementally shifted relative to each other.

7. The method of claim 6, in which performing a plurality of comparisons of the stored sets of comparison data to one another comprises:

quantifying a difference value between the stored sets of comparison data; and

creating a set of local mean unsharp masked difference magnitudes from the quantified difference value.

8. The method of claim 7, further comprising:

generating a mismatch image from the set of local mean unsharp masked difference magnitudes.

9. The method of claim 8, further comprising:

generating a disparity image from the set of local mean unsharp masked difference magnitudes.

10. The method of claim 5, further comprising horizontally downsampling each of the left-eye image and right-eye image into a set of downsampled image lines, or horizontally downsampling the set of downsampled image lines created by the vertical downsampling.

11. The method of claim 5 in which the IIR low pass filter has coefficients a1=1−

b0.

12. The method of claim 5 in which generating a set of difference data comprises filtering the downsampled image line with a bi-directional IIR filter having coefficients a1=1−

b0.

13. An image processing device, comprising:

an input for receiving a stereoscopic video including a left-eye image and a right-eye image;

a difference measurer structured to generate a set of outputs corresponding to a plurality of comparisons between the left-eye image and the right-eye image, the comparisons made at a plurality of relative shifts between the left-eye image and right-eye image; and

a mismatch image creator structured to accept the set of outputs of the difference measurer as an input, to generate a two-dimensional mismatch image, and to assign a color to each region of the mismatch image, in which each region of the mismatch image corresponds to one of the set of outputs and the assigned color is indicative of an amount of relative disparity between portions of the left-eye image and the right-eye image for one of the plurality of relative shifts.

14. The image processing device of claim 13 further comprising a downsampler structured to downsample the left-eye image and the right-eye image.

15. The image processing device of claim 14 in which the downsampler comprises a vertical downsampler having a first downsampling intensity input and a horizontal downsampler having a second downsampling intensity input.

16. The image processing device of claim 13, further comprising:

an unsharp masked video producer structured to accept the left-eye image and the right-eye image as inputs and to generate unsharp masked images as respective outputs.

17. The image processing device of claim 13 in which the unsharp masked video producer comprises a bi-directional IIR filter having coefficients a1=1−

b0.

18. The image processing device of claim 13 in which the mismatch image is scaled to have the same dimensions as images of the stereoscopic video input.

19. The image processing device of claim 13, further comprising a disparity image creator structured to accept the output of the difference measurer as an input and to generate a disparity image having a plurality of image regions that are indicative of a relative disparity between portions of the left-eye image and right-eye image.

20. The image processing device of claim 19 in which the disparity image creator comprises a bi-directional IIR filter having coefficients a1=1−

b0.