Perceptual three-dimensional (3D) video coding based on depth information

• US 10,455,235 B2
• Filed: 10/01/2018
• Issued: 10/22/2019
• Est. Priority Date: 09/29/2011
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Kwon, Do-Kyoung, Budagavi, Madhukar, Chen, Ming-Jun
• Primary Examiner(s): Williams, Jeffery A
• Application Number:
  US16/148,764
• Publication Number:
  US 20190037218A1
• Time in Patent Office: 386 Days
• Field of Search: 37524003
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A method for encoding a multi-view frame in a video encoder is provided that includes computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame, computing a depth-based perceptual quantization scale for a 2D coding block of the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the 2D frame including the 2D coding block, and encoding the 2D coding block using the depth-based perceptual quantization scale.

Claims

1. A method comprising:

receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;

computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;

computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and

encoding the 2D coding block using the depth-based perceptual quantization scale.

2. The method of claim 1, wherein the 2D coding block is a macroblock.

3. The method of claim 1, wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.

4. The method of claim 1, wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.

5. The method of claim 4, wherein the function is f(d,d₀)=c₀·

|d−

d₀|+c₁, wherein c₀ and c₁ are tuning parameters, d is the depth of the multi-view coding block, and d₀ is the depth with the least quality sensitivity.

6. The method of claim 5, wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.

7. The method of claim 5, wherein the depth with the least quality sensitivity is user-specified.

8. The method of claim 1, wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.

9. The method of claim 8, wherein the depth-based perceptual scaling factor is computed as

10. An apparatus comprising:

Means for receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;

means for computing a depth quality sensitivity measure for a multi-view coding block in the multi-view frame based in part on the received depth map;

means for computing a depth-based perceptual quantization scale the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and

means for encoding the 2D coding block using the depth-based perceptual quantization scale.

11. The apparatus of claim 10, wherein the 2D coding block is a macroblock.

12. The apparatus of claim 10, wherein the multi-view frame is a stereoscopic frame comprising a left 2D frame and a right 2D frame.

13. The apparatus of claim 10, wherein the means for computing a depth quality sensitivity measure comprises means for computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.

14. The apparatus of claim 10, wherein the function is f(d,d₀)=c₀·

|d−

d₀|+c₁, wherein c₀ and c₁ are tuning parameters, d is the depth of the multi-view coding block, and d₀ is the depth with the least quality sensitivity.

15. The apparatus of claim 14, wherein the depth with the least quality sensitivity is the farthest depth of a multi-view coding block in the multi-view frame.

16. The apparatus of claim 14, wherein the depth with the least quality sensitivity is user-specified.

17. The apparatus of claim 10, wherein computing a depth-based perceptual quantization scale comprises applying a depth-based perceptual scaling factor to the base quantization scale, wherein the depth-based perceptual scaling factor is determined based on the depth quality sensitive measure, an average of depth quality sensitivity measures of all multi-view coding blocks in the multi-view frame, and a number of multi-view coding blocks in the multi-view frame.

18. The apparatus of claim 17, wherein the depth-based perceptual scaling factor is computed as

19. A non-transitory computer-readable medium storing software instructions that, when executed by a processor, perform a method, the method comprising:

receiving a depth map for a multi-view frame, the multi-view frame comprises a multi-view coding block and the multi-view coding block comprises a 2D coding block;

computing a depth quality sensitivity measure for the multi-view coding block in the multi-view frame based in part on the received depth map;

computing a depth-based perceptual quantization scale for the multi-view coding block, wherein the depth-based perceptual quantization scale is based on the depth quality sensitive measure and a base quantization scale for the multi-view frame; and

encoding the 2D coding block using the depth-based perceptual quantization scale.

20. The computer readable medium of claim 19, wherein computing a depth quality sensitivity measure comprises computing the depth quality sensitivity measure as a function of a depth of the multi-view coding block and a depth with a least quality sensitivity.