STEREO-IMAGE QUALITY AND DISPARITY/DEPTH INDICATIONS

• US 20120249750A1
• Filed: 12/13/2010
• Published: 10/04/2012
• Est. Priority Date: 12/15/2009
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Izzat, Izzat, Li, Feng
• Granted Patent: US 9,030,530 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/144 for flicker reduction
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices

Abstract

A variety of implementations are described. At least one implementation modifies one or more images from a stereo-image pair in order to produce a new image pair that has a different disparity map. The new disparity map satisfies a quality condition that the disparity of the original image pair did not. In one particular implementation, a first image and a second image that form a stereo image pair are accessed. A disparity map is generated for a set of features from the first image that are matched to features in the second image. The set of features is less than all features in the first image. A quality measure is determined based on disparity values in the disparity map. The first image is modified, in response to the determined quality measure, such that disparity for the set of features in the first image is also modified.

Claims

1-21. -21. (canceled)

22. A method comprising:

accessing a first image and a second image that form a stereo image pair;

generating a disparity map for a set of features from the first image that are matched to features in the second image;

determining multiple quality measures based on disparity values from the disparity map for less than all features in the first image, the multiple quality measures including maximum disparity for the stereo image pair and minimum disparity for the stereo image pair, and the multiple quality measures each having a corresponding target quality range;

determining that at least one of the multiple quality measures is outside of the corresponding target quality range; and

modifying at least the first image relative to the second image, such that disparity for the set of features in the first image is also modified, and the at least one quality measure is moved into the corresponding target quality range.

23. The method of claim 22 wherein modifying at least the first image results in all of the multiple quality measures being within the corresponding target quality ranges.

24. The method of claim 22 wherein modifying at least the first image comprises shifting at least the first image relative to the second image, such that the disparity for the set of features in the first image is also modified, and the at least one quality measure is moved into the corresponding target quality range.

25. The method of claim 22 wherein the disparity map is sparse.

26. The method of claim 22 wherein generating the disparity map comprises using a feature matching algorithm.

27. The method of claim 26 wherein the feature matching algorithm comprises at least one of SIFT, k-d tree analysis, or RANSAC.

28. The method of claim 22 wherein generating the disparity map comprises using a median filter to filter disparity values.

29. The method of claim 28 wherein the median filter is applied to disparity values for a particular object across all frames from a sequence.

30. The method of claim 22 wherein the quality measure is for at least one of a frame or a sequence.

31. The method of claim 22 wherein the quality measure comprises at least one of rate of change of maximum disparity, rate of change of minimum disparity.

32. The method of claim 22 wherein modifying the first image, in response to the determined quality measure, comprises determining that the quality measure does not satisfy at least one of a user preference or a capability of a display device.

33. The method of claim 22 wherein modifying the first image comprises using an additional image, different from the first image and the second image, and wherein the additional image includes one or more features from the set of features.

34. The method of claim 33 wherein the stereo image pair is higher resolution than the additional image, and modifying the first image using the additional image comprises:

generating a lower resolution stereo image pair by downsampling the stereo image pair;

generating a low resolution disparity map using the downsampled stereo image pair and the additional image;

generating a high resolution disparity map by upsampling the low resolution disparity map using the higher resolution stereo image pair; and

rendering a modified first image based on the high resolution disparity map and the determined quality measure.

35. An apparatus comprising:

means for accessing a first image and a second image that form a stereo image pair;

means for generating a disparity map for a set of features from the first image that are matched to features in the second image;

means for determining multiple quality measures based on disparity values from the disparity map for less than all features in the first image, the multiple quality measures including maximum disparity for the stereo image pair and minimum disparity for the stereo image pair, and the multiple quality measures each having a corresponding target quality range;

means for determining that at least one of the multiple quality measures is outside of the corresponding target quality range; and

means for modifying at least the first image relative to the second image, such that disparity for the set of features in the first image is also modified, and the at least one quality measure is moved into the corresponding target quality range.

36. An apparatus, comprising one or more processing devices collectively configured to perform at least the following:

accessing a first image and a second image that form a stereo image pair;

generating a disparity map for a set of features from the first image that are matched to features in the second image;

determining multiple quality measures based on disparity values from the disparity map for less than all features in the first image, the multiple quality measures including maximum disparity for the stereo image pair and minimum disparity for the stereo image pair, and the multiple quality measures each having a corresponding target quality range;

determining that at least one of the multiple quality measures is outside of the corresponding target quality range; and

modifying at least the first image relative to the second image, such that disparity for the set of features in the first image is also modified, and the at least one quality measure is moved into the corresponding target quality range.

37. The apparatus of claim 36 wherein the one or more devices comprises (i) one or more processors, (ii) one or more encoders, or (iii) one or more decoders.

38. The apparatus of claim 36 wherein the one or more devices comprises one or more of a set-top box, a cell-phone, a computer, or a PDA.

39. The apparatus of claim 36 further comprising:

a stereo camera for capturing the stereo image pair; and

an additional camera, different from the stereo camera, for capturing an additional image, wherein the additional image includes one or more features from the set of features, and

wherein modifying the first image comprises using the additional image.

40. The apparatus of claim 39 wherein the stereo camera is higher resolution than the additional camera.

41. A processor readable medium having stored thereon instructions for causing a processor to perform at least the following:

accessing a first image and a second image that form a stereo image pair;

generating a disparity map for a set of features from the first image that are matched to features in the second image;

determining multiple quality measures based on disparity values from the disparity map for less than all features in the first image, the multiple quality measures including maximum disparity for the stereo image pair and minimum disparity for the stereo image pair, and the multiple quality measures each having a corresponding target quality range;

determining that at least one of the multiple quality measures is outside of the corresponding target quality range; and

modifying at least the first image relative to the second image, such that disparity for the set of features in the first image is also modified, and the at least one quality measure is moved into the corresponding target quality range.