Enhancement of stereoscopic effect of an image through use of modified depth information

• US 9,317,957 B2
• Filed: 01/18/2013
• Issued: 04/19/2016
• Est. Priority Date: 01/26/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Okada, Shintaro, Nagano, Takahiro, Hosokawa, Kenichiro
• Primary Examiner(s): Harandi, Siamak
• Application Number:
  US13/744,508
• Publication Number:
  US 20130195347A1
• Time in Patent Office: 1,187 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity

Abstract

An image processing apparatus includes a modifying unit configured to modify depth information representing depths in individual pixels of an image in accordance with content included in the image, thereby generating modified depth information, and an enhancing unit configured to perform a stereoscopic effect enhancement process of enhancing a stereoscopic effect of the image by using the modified depth information generated by the modifying unit.

Claims

1. An image processing apparatus comprising:

one or more processors configured to;

modify depth information representing depths in individual pixels of an image in accordance with content included in the image, thereby generating modified depth information;

perform a stereoscopic effect enhancement process of enhancing a stereoscopic effect of the image by using the modified depth information; and

classify each of the individual pixels of the image into one of a plurality of classes in accordance with a pattern of pixel values of one or more sets of the individual pixels and the modified depth information.

2. The image processing apparatus according to claim 1, wherein the one or more processors are further configured to:

determine the content included in the image.

3. The image processing apparatus according to claim 1, wherein the one or more processors are configured to determine a type of scene of the image, and

wherein the depth information is modified in accordance with the determined type of scene.

4. The image processing apparatus according to claim 3, wherein the one or more processors are configured to modify the depth information by combining the depth information and subsidiary depth information which is predefined for each type of scene.

5. The image processing apparatus according to claim 1, wherein the one or more processors are configured to:

determine a face region of a person in the image, and

modify a depth of the determined face region.

6. The image processing apparatus according to claim 1, wherein the one or more processors are configured to:

determine a region forming a target object in the image, and

modify a depth of the determined region forming the target object.

7. The image processing apparatus according to claim 1, wherein the one or more processors are configured to:

determine an edge and texture in the image, and

modify a depth of a region forming the determined edge or a region forming the determined texture.

8. The image processing apparatus according to claim 1, wherein the one or more processors are configured to enhance the stereoscopic effect of the image by using a visual characteristic related to an atmospheric perspective method.

9. The image processing apparatus according to claim 1, wherein the one or more processors are configured to enhance the stereoscopic effect of the image so that at least one of parameters including contrast, sharpness, brightness, saturation, and resolution increases for a pixel having a smaller depth represented by the modified depth information.

10. The image processing apparatus according to claim 1, wherein the one or more processors are configured to enhance the stereoscopic effect of the image so that at least one of parameters including contrast, sharpness, brightness, saturation, and resolution decreases for a pixel having a larger depth represented by the modified depth information.

11. The image processing apparatus according to claim 1, wherein the one or more processors are configured to enhance the stereoscopic effect of the image by performing filtering on the image by using a filter coefficient which is determined in accordance with a depth in a preliminary learning process.

12. The image processing apparatus according to claim 1, wherein the one or more processors are configured to:

perform filtering on a filter tap including a pixel and surrounding pixels of the individual pixels by using a filter coefficient, which is determined for each of the plurality of classes in accordance with a depth in a preliminary learning process.

13. The image processing apparatus according to claim 12, wherein the filter coefficient is determined for each of the plurality of classes in the preliminary learning process which is performed by using a plurality of sets,

wherein each of the plurality of sets includes a student image and a teacher image corresponding to the student image, the teacher image having a stereoscopic effect enhanced by using a visual characteristic related to an atmospheric perspective method.

14. The image processing apparatus according to claim 1, wherein the image corresponds to one or both of a right-eye image and a left-eye image constituting a stereoscopic image, and

wherein the image processing apparatus reproduces the stereoscopic image by causing a display device to display the right-eye image and the left-eye image which include the image having the enhanced stereoscopic effect.

15. An image processing method comprising:

in an information processing device comprising one or more processors;

modifying depth information representing depths in individual pixels of an image in accordance with content included in the image, thereby generating modified depth information;

performing a stereoscopic effect enhancement process of enhancing a stereoscopic effect of the image by using the modified depth information; and

classifying each of the individual pixels into one of a plurality of classes in accordance with a pattern of pixel values of one or more sets of the individual pixels and a depth represented by the modified depth information.