Applying perceptually correct 3D film noise

• US 9,031,356 B2
• Filed: 03/12/2013
• Issued: 05/12/2015
• Est. Priority Date: 03/20/2012
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Kunkel, Timo, Chen, Tao, Wan, Chun Chi
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): PERLMAN, DAVID S
• Application Number:
  US13/796,950
• Publication Number:
  US 20130251241A1
• Time in Patent Office: 791 Days
• Field of Search: None
• US Class Current: 382/285
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Perceptually correct noises simulating a variety of noise patterns or textures may be applied to stereo image pairs each of which comprises a left eye (LE) image and a right eye (RE) image that represent a 3D image. LE and RE images may or may not be noise removed. Depth information of pixels in the LE and RE images may be computed from, or received with, the LE and RE images. Desired noise patterns are modulated onto the 3D image or scene so that the desired noise patterns are perceived to be part of 3D objects or image details, taking into account where the 3D objects or image details are on a z-axis perpendicular to an image rendering screen on which the LE and RE images are rendered.

Claims

1. A method comprising:

accessing a left eye (LE) image and a right eye (RE) image that represent a 3D image;

filtering the LE image and the RE image to reduce undesirable noise;

determining, based on depth information relating to the LE and RE image, one or more first depths of a first 3D image feature in the 3D image;

applying, after the filtering, one or more noise patterns to the first 3D image feature in the 3D image based on the one or more first depths of the first 3D image feature in the 3D image;

determining, based on the depth information relating to the LE and RE image, one or more second depths of a second 3D image feature in the 3D image; and

applying the one or more noise patterns to the second 3D image feature in the 3D image based on the one or more second depths of the second 3D image feature in the 3D image;

wherein the one or more noise patterns are applied to the first 3D image feature with one or more first spatial frequency components that are different from one or more second spatial frequency components with which the one or more noise patterns are applied to the second 3D image feature.

2. The method of claim 1, wherein the first 3D image feature is closer to a viewer of the 3D image than the second 3D image feature, and wherein the one or more first spatial frequency components comprise more high spatial frequency components than the one or more second spatial frequency components.

3. The method of claim 1, wherein the first 3D image feature is closer to a viewer of the 3D image than the second 3D image feature, and wherein the one or more first spatial frequency components comprise fewer high spatial frequency components than the one or more second spatial frequency components.

4. A method comprising:

accessing a left eye (LE) image and a right eye (RE) image that represent a 3D image;

filtering the LE image and the RE image to reduce undesirable noise;

determining, based on depth information relating to the LE and RE image, one or more first depths of a first 3D image feature in the 3D image;

applying, after the filtering, one or more noise patterns to the first 3D image feature in the 3D image based on the one or more first depths of the first 3D image feature in the 3D image;

overemphasizing one or more high spatial frequency components in the one or more noise patterns to a 3D image feature in the 3D image, wherein the 3D image feature is closer to a viewer than other 3D image features in the 3D image.

5. The method of claim 4, further comprising:

analyzing the LE and RE images; and

computing the depth information based on results of analyzing the LE and RE images.

6. The method of claim 4, further comprising:

receiving image metadata associated with the LE and RE images; and

retrieving the depth information from the image metadata associated with the LE and RE images.

7. The method of claim 4, wherein the LE and RE images comprise a stereo image pair captured from a reality by a stereoscopic camera system.

8. The method of claim 4, wherein the LE and RE images comprise at least one computer-generated image detail.

9. The method of claim 4, wherein the one or more noise patterns comprises at least one of (a) noise patterns simulating chemical films, analogous chemical properties of silver, chemical film color layers, (b) noise patterns simulating noise characteristics of a specific device;

noise patterns simulating a specific texture, (c) watermarks, (d) noises causing an image feature at a certain depth to go up and down, left and right, or in and out spatially, (e) noises altering luminance or colors of an image feature at a certain depth, (f) noise differentially applied in dark and bright areas;

noises simulating a variety of response curves, saturation properties, noise properties, devices, and media, and (g) noises with one or more of a variety of different distribution patterns or probability distribution functions.

10. The method of claim 4, further comprising applying the one or more noise patterns to a plurality of image features in the 3D images at respective depths of the image features.

11. The method of claim 4, further comprising removing at least a portion of embedded noises in the LE and RE images.

12. The method of claim 4, wherein the one or more noise patterns are applied to the LE and RE images with at least one portion of embedded noises in the LE and RE images remaining in the LE and RE images.

13. The method of claim 4, wherein the LE image and the RE image are selected from an autostereoscopic image that comprises two or more images in different views.

14. An apparatus comprising a processor and configured to perform the method recited in claim 4.

15. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of the method recited in claim 4.

16. A method comprising:

receiving a left eye (LE) image and a right eye (RE) image that represent a 3D image;

determining, based on depth information relating to the LE and RE image, one or more first depths of a first 3D image feature in the 3D image; and

applying one or more noise patterns to the first 3D image feature in the 3D image based on the one or more first depths of the first 3D image feature in the 3D image,

wherein a spatial frequency of the one or more noise patterns is dependent on the depth information;

determining, based on the depth information relating to the LE and RE image, one or more second depths of a second 3D image feature in the 3D image; and

applying the one or more noise patterns to the second 3D image feature in the 3D image based on the one or more second depths of the second 3D image feature in the 3D image;

wherein the one or more noise patterns are applied to the first 3D image feature with one or more first spatial frequency components that are different from one or more second spatial frequency components with which the one or more noise patterns are applied to the second 3D image feature.

17. The method of claim 16, wherein the first 3D image feature is closer to a viewer of the 3D image than the second 3D image feature, and wherein the one or more first spatial frequency components comprise more high spatial frequency components than the one or more second spatial frequency components.

18. The method of claim 16, wherein the first 3D image feature is closer to a viewer of the 3D image than the second 3D image feature, and wherein the one or more first spatial frequency components comprise fewer high spatial frequency components than the one or more second spatial frequency components.

19. A method comprising:

receiving a left eye (LE) image and a right eye (RE) image that represent a 3D image;

determining, based on depth information relating to the LE and RE image, one or more first depths of a first 3D image feature in the 3D image; and

applying one or more noise patterns to the first 3D image feature in the 3D image based on the one or more first depths of the first 3D image feature in the 3D image,

wherein a spatial frequency of the one or more noise patterns is dependent on the depth information;

overemphasizing one or more high spatial frequency components in the one or more noise patterns to a 3D image feature in the 3D image, wherein the 3D image feature is closer to a viewer than other 3D image features in the 3D image.

20. The method of claim 19, wherein the LE image and the RE image are selected from an autostereoscopic image that comprises two or more images in different views.