System and Methods to Efficiently Measure Stereoscopic Disparity and to Generate Related Mismatch Images

• US 20120162364A1
• Filed: 01/11/2011
• Published: 06/28/2012
• Est. Priority Date: 12/28/2010
• Status: Active Grant

• Current Assignee: Project Giants, LLC
• Original Assignee: Tektronix Incorporated (Fortive Corp.)
• Inventors: FERGUSON, KEVIN M.
• Granted Patent: US 9,336,596 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Embodiments of the invention include a system and methods for measuring disparity and mismatch of stereoscopic images of three-dimensional video, which may indicate a level of discomfort on the part of the viewer. Measuring these parameters may also be used to give an indication that certain processes have or have not adversely effected perceived depth (i.e. through conversion from disparity to depth given a display size and viewing distance), verify camera setup or 2-dimension to 3-dimension synthesizing processes. Generating the indications and other data about the stereoscopic images uses techniques that allow processing with much less computing resources than was possible in previous systems.

Claims

1. A method, comprising:

accepting a left and a right image of a stereoscopic image;

creating a set of differential data; and

generating a mismatch image by populating portions of the mismatch image with a color indicative of a level of match of the set of differential data

2. The method of claim 1 in which the left and right images have a height and width, and in which the set of differential data is created from a vertically downsampled version of the left and right images.

3. The method of claim 2 in which each group of data in the set of differential data corresponds to a vertically downsampled image line of the left and right images.

4. The method of claim 3 in which generating a mismatch image comprises generating a line of the mismatch image from one group of the set of differential data, and in which the line of the mismatch image is vertically expanded to fill a region of the mismatch image that corresponds to the region of the stereoscopic image from which the line was created.

5. A method of generating differential data from a stereoscopic image having left and right image components, the method comprising:

vertically downsampling each of the left and right images into a set of downsampled image lines;

creating an unsharp mask for at least one of the downsampled image lines by;

filtering the at least one downsampled image line with an IIR low pass filter to create a filtered downsampled image line, and

removing the filtered downsampled image line components from the at least one downsampled image line.

6. The method of generating differential data of claim 5, further comprising:

generating a set of difference data for the at least one of the downsampled image lines by comparing the downsampled image line to its own unsharp mask and storing the comparison.

7. The method of claim 6, in which the stored comparison is for a left image, the method further comprising:

generating a set of difference data for the at least one of the downsampled image lines in the right image by comparing the downsampled image line to its own unsharp mask and storing the comparison; and

comparing the stored sets of comparison data from the left image and the right image to one another.

8. The method of claim 7, in which comparing the stored sets of comparison data to one another comprises:

quantifying a difference value between the stored sets of comparison data; and

creating a set of local mean unsharp masked difference magnitudes from the quantified difference value.

9. The method of claim 8, further comprising:

generating a mismatch image from the set of local mean unsharp masked difference magnitudes.

10. The method of claim 9, further comprising:

generating a disparity image from the set of local mean unsharp masked difference magnitudes.

11. The method of claim 6, further comprising horizontally downsampling each of the left and right images into a set of downsampled image lines, or horizontally downsampling the set of downsampled image lines created by the vertical downsampling.

12. The method of claim 5 in which the IIR low pass filter has coefficients a1=1−

b0.

13. The method of claim 7 in which generating a set of difference data comprises filtering the downsampled image line with a bi-directional IIR filter having coefficients a1=1−

b0.

14. An image processing device, comprising:

an input for receiving a stereoscopic video including a left image and a right image;

a difference measurer structured to generate an output indicative of a plurality of comparisons between the left image and the right image, the comparisons made at a plurality of relative shifts between the left image and right image;

a mismatch image creator structured to accept the output of the difference measurer as an input and to generate a mismatch image having a plurality of image regions that are indicative of an amount of relative disparity between portions of the left image and right image.

15. The image processing device of claim 14 further comprising a downsampler structured to downsample the left image and the right image.

16. The image processing device of claim 15 in which the downsampler comprises a vertical downsampler having a first downsampling intensity input and a horizontal downsampler having a second downsampling intensity input.

17. The image processing device of claim 14, further comprising:

an unsharp masked video producer structured to accept the left image and the right image as inputs and to generate unsharp masked images as respective outputs.

18. The image processing device of claim 14 in which the unsharp masked video producer comprises a bi-directional IIR filter having coefficients a1=1−

b0.

19. The image processing device of claim 14 in which the mismatch image is scaled to have the same dimensions as images of the stereoscopic video input.

20. The image processing device of claim 14, further comprising a disparity image creator structured to accept the output of the difference measurer as an input and to generate a disparity image having a plurality of image regions that are indicative of a relative disparity between portions of the left image and right image

21. The image processing device of claim 20 in which the disparity image creator comprises a bi-directional IIR filter having coefficients a1=1−

b0.