METHOD AND SYSTEM FOR RE-PROJECTION FOR MULTIPLE-VIEW DISPLAYS

• US 20140210810A1
• Filed: 01/31/2013
• Published: 07/31/2014
• Est. Priority Date: 01/31/2013
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Technologies Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Pictures Technologies Incorporated (Sony Group Corp.)
• Inventors: Davidson, Alan L., Havaldar, Parag, Palombi, Peter H.
• Granted Patent: US 10,931,927 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

Systems and methods are provided that allow for the rapid creation of multiple novel views for applications. These multiple views may allow for not only stereo views but also for different viewpoints of a stereographic scene to be presented, or for a time progression of the stereo scene to be presented, where a viewpoint is altered with respect to time. The systems and methods may be employed to create the content for auto-stereographic displays or multiple view systems that use a plurality of novel views to display stereo media. Such displays or multiple view systems may include those employing parallax barriers or lenticular lenses.

Claims

1. A system for creating a multiple-view display, comprising:

an input module for receiving a computer-generated image of a scene, the computer-generated image including at least depth information about the scene;

a re-projection module for creating a plurality of alternate images corresponding to different views of the scene, wherein the image and alternate images are such that a viewer observing the image and one of the plurality, or two of the plurality, perceives a view of the scene as a three-dimensional scene; and

a storage module for storing the image and the plurality of images.

2. The system of claim 1, wherein the image is from a video file.

3. The system of claim 1, wherein the image is received from a game engine.

4. The system of claim 1, wherein the re-projection module selects a location for the alternate image, creates a disparity map indicating differences between the image and the alternate image, creates a distortion map indicating pixel transforms based on the disparity map, and creates the alternate image by applying the distortion map to the pixels of the image.

5. The system of claim 4, wherein the re-projection module further:

applies a custom adaptive sharpening filter to one or more objects in the alternate image, the custom adaptive sharpening filter configured to increase a prevalence of high-frequency components and decrease a prevalence of low-frequency components;

or

for one or more objects in the image, the one or more objects surrounded by a background or clipping plane, temporarily extends the object'"'"'s size by 1-10 pixels whereby when re-projection occurs, pixels in the object are mapped properly during re-projection and not extended back to the background or clipping plane.

6. A method for creating a multiple-view display, comprising:

receiving a computer-generated image of a scene, the computer-generated image having information including at least depth information;

using a step of re-projection, generating a plurality of alternate images, the alternate images corresponding to different views of the scene, wherein the image and alternate images are such that a viewer observing the image and one of the plurality, or two of the plurality, perceives a view of the scene as a three-dimensional scene; and

storing the image and the alternate images corresponding to the different views of the scene.

7. The method of claim 6, wherein the plurality numbers from two to sixteen.

8. The method of claim 7, wherein the plurality includes eight images, four on one side of the image and four on an opposite side of the image.

9. The method of claim 8, wherein each set of four are equi-angularly spaced from the image.

10. The method of claim 6, wherein the step of re-projection includes calculating a view of the scene from a different angle.

11. The method of claim 10, wherein the step of re-projection includes:

selecting a location for the alternate image;

creating a disparity map indicating differences between the image and the alternate image;

creating a distortion map indicating pixel transforms based on the disparity map; and

creating the alternate image by applying the distortion map to the pixels of the image.

12. The method of claim 11, wherein the step of re-projection further includes performing one or both of the following steps:

applying a custom adaptive sharpening filter to one or more objects in the alternate image, the custom adaptive sharpening filter configured to increase a prevalence of high-frequency components and decrease a prevalence of low-frequency components;

or

for one or more objects in the image, the one or more objects surrounded by a background or clipping plane, temporarily extending the object'"'"'s size by 1-10 pixels whereby when re-projection occurs, pixels in the object are mapped properly during re-projection and not extended back to the background or clipping plane.

13. A multiple-view display, comprising:

an input module for receiving a data file, the data file including multiple sets of images corresponding to various views of a scene, each set corresponding to a geographic area on a display screen, each member of each set corresponding to a different view of the scene as displayed at the geographic area, at least one member of each set developed by a step of re-projection of an original image, the original image including at least depth information about the scene;

a display module to display each set at its corresponding geographic area, and to display each member of the set within the area;

a directional viewing system disposed adjacent the display module, the directional viewing system configured to direct a view of each member substantially at a given angle relative to the plane of the geographic area.

14. The display of claim 13, wherein the directional viewing system is a lenticular display or a parallax barrier.

15. The display of claim 13, wherein the given angle is different for each member of a set.

16. The display of claim 13, wherein corresponding members of sets direct views at a common angle relative to the plane of the geographic area.

17. The display of claim 13, wherein the directional viewing system is configured to direct views at between two and sixteen different angles.

18. The display of claim 13, wherein the geographic area is a line of display.

19. The display of claim 13, wherein the geographic area is in the shape of a rectangle.

20. The display of claim 13, wherein each member of the set is created by re-projection by selecting a location for the member of the set, creating a disparity map indicating differences between the original image and an image corresponding to the member of the set at the location, creating a distortion map indicating pixel transforms based on the disparity map, and creating the image corresponding to the member of the set at the location by applying the distortion map to the pixels of the original image.

21. The display of claim 20, wherein the image corresponding to the member of the set is further created by:

applying a custom adaptive sharpening filter to one or more objects in the image corresponding to the member of the set, the custom adaptive sharpening filter configured to increase a prevalence of high-frequency components and decrease a prevalence of low-frequency components;

or

for one or more objects in the original image, the one or more objects surrounded by a background or clipping plane, temporarily extending the object'"'"'s size by 1-10 pixels whereby when re-projection occurs, pixels in the object are mapped properly during re-projection and not extended back to the background or clipping plane.

22. A data file, the data file including multiple sets of images corresponding to various views of a scene, each set corresponding to a geographic area on a display screen, each member of each set corresponding to a different view of the scene as displayed at the geographic area, at least one member of each set developed by a step of re-projection of an original image, the original image including at least depth information about the scene.

23. The data file of claim 22, wherein the geographic area is a vertical line of display.

24. The data file of claim 22, wherein the geographic area is in the shape of a rectangle.

25. The data file of claim 22, wherein each member of the set is created by re-projection by selecting a location for the member of the set, creating a disparity map indicating differences between the original image and an image corresponding to the member of the set at the location, creating a distortion map indicating pixel transforms based on the disparity map, and creating the image corresponding to the member of the set at the location by applying the distortion map to the pixels of the original image.

26. The data file of claim 25, wherein the image corresponding to the member of the set is further created by:

applying a custom adaptive sharpening filter to one or more objects in the image corresponding to the member of the set, the custom adaptive sharpening filter configured to increase a prevalence of high-frequency components and decrease a prevalence of low-frequency components;

or

for one or more objects in the original image, the one or more objects surrounded by a background or clipping plane, temporarily extending the object'"'"'s size by 1-10 pixels whereby when re-projection occurs, pixels in the object are mapped properly during re-projection and not extended back to the background or clipping plane.