APPARATUS AND METHOD FOR GENERATING AN IMAGE

• US 20200137375A1
• Filed: 01/03/2019
• Published: 04/30/2020
• Est. Priority Date: 06/29/2017
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: KROON, BART, DE HAAN, WIEBE
• Granted Patent: US 11,218,690 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/302 for viewing without the aid of special glasses, i.e. using autostereoscopic displays

Abstract

An apparatus for generating an image comprises a receiver (101) which receives 3D image data providing an incomplete representation of a scene. A view vector source (103) provides a rendering view vector indicative of a rendering viewpoint for the image. A renderer (105) renders a first region of an intermediate image for the rendering view vector based on image data from the 3D image data. An extrapolator (107) extrapolates the 3D image data into a second adjoining region of the intermediate image with the 3D image not comprising image data for the second region. A blur processor (109) generates the image in response to applying a spatially varying blurring to the intermediate image where a degree of blurring is higher in a transitional region between the first region and the second region than in an internal area of the first region.

Claims

1. An apparatus for generating an image, the apparatus comprising:

a first receiver circuit, the first receiver circuit arranged to receive a 3D image data of a scene, wherein the 3D image data provides an incomplete representation of the scene in that the 3D image data does not provide a description of the visual properties for at least one part of the scene for at least one viewpoint;

a view vector source, the view vector source arranged to provide a rendering view vector indicative of a first viewpoint in the scene;

a renderer circuit, the renderer circuit arranged to render a first region of a first image based on the 3D image data,

wherein the first image is for the first viewpoint indicated by the rendering view vector,

wherein the 3D image data comprises image data for the first region;

an extrapolator circuit, the extrapolator circuit arranged to extrapolate the 3D image data into a second region of the first image, wherein the 3D image data does not comprise image data for the second region and the first region and the second region are adjoining regions; and

a blur processor circuit, wherein the blur processor circuit generates the image,

wherein generating the image comprises applying a spatially varying blurring to the first image,

wherein a degree of blurring is higher in a transitional region between the first region and the second region than for an internal area of the first region.

2. The apparatus of claim 1, wherein the transitional region includes an area of the first region.

3. The apparatus of claim 1, further comprising:

a render region processor circuit, the render region processor circuit arranged to determine a rendering mask for the image, wherein the rendering mask is indicative of whether pixels of the image belong to the first region or not;

a filter adapter circuit, the filter adapter circuit arranged to determine a spatially varying filter characteristic for the image in response to the rendering mask, wherein the blur processor circuit is arranged to apply a spatial filter adopting the spatially varying filter characteristic to the rendered image.

4. The apparatus of claim 3, wherein the spatially varying filter characteristic is indicative of a degree of low pass filtering by the spatial filter.

5. The apparatus of claim 3, wherein the extrapolator circuit is arranged to:

generate a downscaled image in response to downscaling the first image;

generate a filtered downscaled image in response to applying a bilateral filter to the downscaled image, wherein the bilateral filter is guided by the rendering mask;

generate a filtered upscaled image in response to upscaling the downscaled image; and

generate pixel values for the second region from the filtered upscaled image.

6. The apparatus of claim 1, claim wherein the blur processor circuit is arranged to determine the degree of blurring for a pixel in response to a distance from a position of the pixel to a border of the transition area.

7. The apparatus of claim 1, wherein the blur processor circuit is arranged to reduce a degree of blurring in at least part of the second region with respect to a degree of blurring in the transitional region.

8. The apparatus of claim 1, further comprising:

a second receiver circuit, the second receiver circuit arranged to receive a target view vector indicative of a target viewpoint in the scene for the image;

a reference source, the reference source arranged to provide a reference view vector indicative of a reference viewpoint for the scene,

wherein the view vector source is arranged to generate the rendering view vector indicative of a rendering viewpoint as a function of the user desired viewpoint and the reference viewpoint for the scene.

9. The apparatus of claim 8, wherein the view vector source is arranged to determine the reference view vector in response to a border view vector indicative of a border between a part of the scene for which the 3D image data comprises image data and a part of the scene for which the 3D image data does not comprise image data.

10. The apparatus of claim 8, wherein the first receiver circuit is arranged to receive a data stream comprising:

the 3D image data; and

nominal viewpoint data, wherein the view vector source is arranged to generate the reference view vector in response to the nominal viewpoint data.

11. The apparatus of claim 1,

wherein the extrapolator circuit is arranged to determine a low pass filtered image property for a plurality of areas of the first region proximal to the second region,

wherein the extrapolator circuit is arranged to set an image property for at least part of the second region in response to the low pass filtered image properties.

12. The apparatus of claim 1,

wherein the first receiver circuit is arranged to receive the 3D image data in a data stream,

wherein the data stream comprise extrapolation control data,

wherein the extrapolator circuit is arranged to set a parameter of the extrapolation in response to the extrapolation control data.

13. The apparatus of claim 1, further comprising a filter processor circuit, the filter processor circuit arranged to apply a global spatial filter to the image, wherein a property of the spatial filter is dependent on a proportion of the image for which the 3D image does not comprise image data.

14-1. A computer program stored on a non-transitory medium, wherein the computer program when executed on processor performs the method as claimed in claim 13.

15. The method of claim 13, wherein the transitional region includes an area of the first region.

16. The method of claim 13, further comprising:

determining a rendering mask for the image, wherein the rendering mask is indicative of whether pixels of the image belong to the first region or not;

determining a spatially varying filter characteristic for the image in response to the rendering mask, wherein the blurring is arranged to apply a spatial filter adopting the spatially varying filter characteristic to the rendered image.

17. The method of claim 16, wherein the spatially varying filter characteristic is indicative of a degree of low pass filtering by the spatial filter.

18. The method of claim 16, wherein the extrapolating comprises:

generating a downscaled image in response to downscaling the first image;

generating a filtered downscaled image in response to applying a bilateral filter to the downscaled image, wherein the bilateral filter is guided by the rendering mask;

generating a filtered upscaled image in response to upscaling the downscaled image; and

generating pixel values for the second region from the filtered upscaled image.

19. The method of claim 13, wherein the blurring circuit determines the degree of blurring for a pixel in response to a distance from a position of the pixel to a border of the transition area.

20. The method of claim 13, wherein the blurring reduces a degree of blurring in at least part of the second region with respect to a degree of blurring in the transitional region.

14. A method of generating an image, the method comprising:

receiving a 3D image data for a scene, wherein the 3D image data provides an incomplete representation of the scene in that the 3D image data does not provide a description of the visual properties for at least one part of the scene for at least one viewpoint;

providing a rendering view vector indicative of a first viewpoint in the scene;

rendering a first region of a first image based on the 3D image data,

wherein the first image is for the first viewpoint indicated by the rendering view vector,

wherein the 3D image data comprising image data for the first region;

extrapolating the 3D image data into a second region of the first image, wherein the 3D image data does not comprise image data for the second region and the first and second regions are adjoining regions; and

generating the image,

wherein generating the image comprises applying a spatially varying blurring to the first image,

wherein a degree of blurring is higher in a transitional region between the first region and the second region than for an internal area of the first region.