SALIENCY BASED DISPARITY MAPPING

• US 20140313191A1
• Filed: 10/23/2012
• Published: 10/23/2014
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Bruls, Wilhelmus Hendrikus Alfonsus, Vandewalle, Patrick Luc Els, Koon, Bart
• Granted Patent: US 9,041,709 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity

Abstract

A three dimensional [3D] image signal is processed for rendering 3D image data (33) on a specific 3D display, e.g. an auto-stereoscopic display. A first depth map (34) and saliency of the 3D image data are determined. A display depth sub-range (35) of a usable depth range (36) of the 3D display is determined and provides a higher 3D image quality for a viewer than the 3D image quality across the usable depth range. A depth mapping function is determined in dependence of the saliency data. The depth mapping function maps the first depth map to a second depth map for generating views for the 3D display. Advantageously the salient range of depth values is mapped towards the display depth sub-range.

Claims

1. Method of processing a three dimensional [3D] image signal for rendering 3D image data on a 3D display based on multiple display views, the method comprising:

deriving, from the 3D image signal, a first depth map comprising first depth values,

deriving, from the 3D image signal, saliency data representing saliency in an amount of the 3D image data, the saliency being indicative of salient elements having a salient range of depth values,

determining a depth mapping function for the amount of the 3D image data in dependence of the saliency data such that first depth values in the salient range of depth values are mapped towards a display depth sub-range, the display depth sub-range being a sub-range of a usable depth range of the 3D display and providing a higher 3D image quality for a viewer than the 3D image quality across the usable depth range, and wherein determining the depth mapping function comprises;

making a combined histogram for salient depth values and non-salient depth values, wherein the salient depth values are weighted using a first input conversion weight before adding in the histogram and the non-salient depth values are weighted using a second input conversion weight before adding in the histogram, the first input conversion weight and the second input conversion weight in dependence of the display depth sub-range and the usable range of the 3D display,

determining a combined histogram depth range, and

mapping the combined histogram depth range inside the usable depth range of the 3D display, the first input conversion weight and the second input conversion weight being different and selected so that the first depth values in the salient range of depth values are warped towards the display depth sub-range,

converting the first depth map into a second depth map by the depth mapping function for generating the display views in dependence of the second depth map.

2. Method as claimed in claim 1, wherein

the saliency data is a saliency map indicating a degree of saliency for pixels of the image, or

the saliency data is a saliency depth range.

3. Method as claimed in claim 1, wherein deriving saliency data comprises deriving meta data from the 3D image signal, the meta data representing the saliency data of the 3D image data.

6. Method as claimed in claim 1, wherein the amount of 3D image data is a period of a 3D video signal, and the depth mapping function depends on the saliency data across time.

7. Method as claimed in claim 6, wherein the depth mapping function comprises

determining a shot as the period of the 3D video signal and setting the mapping function for the shot, or

using a moving window having a look-ahead and setting the mapping function for the window.

14. Computer program comprising computer program code means operative to cause a processor to perform the respective steps of the method according to claim 1 when the computer program is run on a computer.

15. Computer readable medium comprising the computer program as claimed in claim 14.

4. (canceled)

5. (canceled)

8. 3D video device for processing a three dimensional [3D] image signal for rendering 3D image data on a 3D display based on multiple display views, the device comprising:

input means for

deriving, from the 3D image signal, a first depth map comprising first depth values, and for

deriving, from the 3D image signal, saliency data representing saliency in an amount of the 3D image data, the saliency being indicative of salient elements having a salient range of depth values, and

a video processor arranged for

determining a depth mapping function for the amount of the 3D image data in dependence of the saliency data such that first depth values in the salient range of depth values are mapped towards a display depth sub-range, the display depth sub-range being a sub-range of a usable depth range of the 3D display and providing a higher 3D image quality for a viewer than the 3D image quality across the usable depth range, and wherein determining the depth mapping function comprises;

making a combined histogram for salient depth values and non-salient depth values, wherein the salient depth values are weighted using a first input conversion weight before adding in the histogram and the non-salient depth values are weighted using a second input conversion weight before adding in the histogram, the first input conversion weight and the second input conversion weight in dependence of the display depth sub-range and the usable range of the 3D display;

determining a combined histogram depth range, and

mapping the combined histogram depth range inside the usable depth range of the 3D display, the first input conversion weight and the second input conversion weight being different and selected so that the first depth values in the salient range of depth values are warped towards the display depth sub-range,

converting the first depth map into a second depth map by the depth mapping function for generating the display views in dependence of the second depth map.

9. Device as claimed in claim 8, wherein the video processor is arranged for

generating the display views in dependence of the second depth map, and/or

the device comprises a 3D display for displaying the display views for the respective left and right eyes of a viewer, and/or

the video processor is arranged for determining the display depth sub-range by acquiring display capability information from a 3D display device connected to the 3D video device.

10. Device as claimed in claim 8, wherein said deriving saliency data comprises deriving meta data from the 3D image signal, the meta data representing the saliency data of the 3D image data.

11. Device as claimed in claim 8, wherein the input means comprise:

an optical record carrier unit for retrieving the video information from an optical record carrier.

12. (canceled)

13. (canceled)