Method for determining stereo quality score and automatically improving the quality of stereo images

• US 9,530,192 B2
• Filed: 06/30/2011
• Issued: 12/27/2016
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Inventors: Park, Minwoo, Luo, Jiebo, Gallagher, Andrew Charles
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): SECHSER, JILL D
• Application Number:
  US13/173,264
• Publication Number:
  US 20130002814A1
• Time in Patent Office: 2,007 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/144 for flicker reduction

Abstract

A method for improving a stereo image including a left view image and a right view image, comprising: using a data processor to automatically analyze the stereo image to determine an original stereo quality score responsive to relative positions of corresponding points in the left view image and the right view image; specifying a set of one or more candidate modifications to the stereo image; determining revised stereo quality scores based on each of the candidate modifications to the stereo image; selecting a particular candidate modification that produces a revised stereo quality score which indicates a higher quality level than the original stereo quality score; forming an output stereo image corresponding to the selected particular candidate modification; and storing the output stereo image in a processor-accessible memory.

Claims

1. A method for improving a stereo image including a left view image and a right view image, comprising:

using a digital processor to automatically analyze the stereo image to determine an original stereo quality score responsive to relative positions of corresponding points in the left view image and the right view image;

specifying a set of one or more candidate modifications to the stereo image;

determining revised stereo quality scores based on each of the candidate modifications to the stereo image;

generating a plurality of iterated revised stereo quality scores and a new set of modifications until either the iterated revised stereo quality score does not change or until an iteration limit is reached;

selecting a particular candidate modification that produces a revised stereo quality score which indicates a higher quality level than the original stereo quality score;

forming an output stereo image corresponding to the selected particular candidate modification; and

storing the output stereo image in a processor-accessible memory.

2. The method of claim 1 wherein the candidate modifications include applying a geometric transform to the left view image or the right view image or both.

3. The method of claim 2 wherein the geometric transform includes a translation, a rotation or a scaling operation.

4. The method of claim 2 wherein different geometric transforms are applied to different image regions.

5. The method of claim 1 wherein the set of one or more candidate modifications are determined using an iterative optimization process.

6. The method of claim 1 wherein each candidate modification is specified using one or more parameters.

7. The method of claim 6 wherein the set of one or more candidate modifications correspond to a systematic sampling of the one or more parameters.

8. The method of claim 7 wherein the systematic sampling involves selecting random values for the one or more parameters according to a predefined distribution.

9. The method of claim 7 wherein the systematic sampling involves selecting values for the one or more parameters according to a regular pattern.

10. The method of claim 1 wherein the selected particular candidate modification is the candidate modification that produces the revised stereo quality score corresponding to the highest quality level among the one or more candidate modifications.

11. The method of claim 1 wherein the process of determining the stereo quality score includes:

extracting a set of features from the stereo image, wherein the features include a set of motion vectors between corresponding points in the left view image and the right view image;

determining the stereo quality score using a classifier responsive to the set of features.

12. The method of claim 11 wherein the classifier is trained using a training set of stereo images having reference stereo quality scores determined by human observers.

13. The method of claim 11 wherein the set of motion vectors are determined by tracking a set of interest points between the left view image and the right view image.

14. The method of claim 13 wherein the set of interest points are tracked using a Kanade-Lucas-Tomasi feature tracking algorithm.

15. The method of claim 11 wherein the set of features further includes one or more features incorporating epipolar geometry characteristics determined from the set of motion vectors.

16. The method of claim 11 wherein the set of features further includes one or more features determined by applying statistical calculations to the set of motion vectors.

17. The method of claim 11 wherein the classifier is a support vector machine classifier.

18. The method of claim 1 further including displaying the stereo image pair using a stereo image display system.