Method for modeling and estimating rendering errors in virtual images

• US 8,692,840 B2
• Filed: 02/05/2012
• Issued: 04/08/2014
• Est. Priority Date: 02/05/2012
• Status: Active Grant

• Current Assignee: Bishi Electric Research Laboratories, Inc.
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Cheung, Ngai-Man, Tian, Dong, Vetro, Anthony, Sun, Huifang
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US13/366,348
• Publication Number:
  US 20130201177A1
• Time in Patent Office: 793 Days
• Field of Search: 345418-419, 345/422, 345/428, 345581-582, 345/586, 345/589, 345/606, 345/610, 345/616, 345618-619, 345629-630, 345/643, 382/154, 382/252, 382/254, 382274-276, 382/284, 382/285, 463 31- 32, 715/273, 715/700, 715/706, 715/757, 715763-764, 715781-782, 715848-852
• US Class Current: 345/582
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection

Abstract

A quality of a virtual image for a synthetic viewpoint in a 3D scene is determined. The 3D scene is acquired by texture images, and each texture image is associated with a depth image acquired by a camera arranged at a real viewpoint. A texture noise power is based on the acquired texture images and reconstructed texture images corresponding to a virtual texture image. A depth noise power is based on the depth images and reconstructed depth images corresponding to a virtual depth image. The quality of the virtual image is based on a combination of the texture noise power and the depth noise power, and the virtual image is rendered from the reconstructed texture images and the reconstructed depth images.

Claims

1. A method for estimating a quality of a virtual image for a synthetic viewpoint in a 3D scene, wherein the 3D scene is acquired by a plurality of texture images, and each texture image is associated with a depth image, comprising the steps of:

determining a texture noise power based on the plurality of acquired texture images and a plurality of reconstructed texture images corresponding to a virtual texture image;

determining a depth noise power based on the plurality of acquired depth images and a plurality of reconstructed depth images corresponding to a virtual depth image; and

estimating the quality of the virtual image based on a combination of the texture noise power and the depth noise power, wherein the virtual image is rendered from the plurality of reconstructed texture images and the plurality of reconstructed depth images, wherein the steps are performed in a processing device.

2. The method of claim 1, wherein the texture noise power is based on a mean squared error between the plurality of acquired texture images and the plurality of reconstructed texture images.

3. The method of claim 1, wherein the rendering of the virtual image blends image samples from the plurality of reconstructed texture images according to the plurality of reconstructed depth images.

4. The method of claim 3, wherein the blending is linear and uses weights that are proportional to a distance between the synthetic viewpoint and the real viewpoint of each texture image.

5. The method of claim 1, wherein the texture noise power and the depth noise power are determined independently.

6. The method of claim 1, wherein a correlation coefficient is a difference between the acquired texture image and the reconstructed texture image.

7. The method of claim 6 wherein the correlation coefficient is determined by averaging the texture noise power of the acquired texture image and the reconstructed texture image.

8. The method of claim 1, wherein the depth noise power is a product of an integration of a power spectral density and a frequency envelope.

9. The method of claim 1, wherein the 3D scene is comprised of two viewpoints including a left viewpoint and a right viewpoint.