Synthesizing Panoramic Three-Dimensional Images

• US 20110242271A1
• Filed: 03/31/2010
• Published: 10/06/2011
• Est. Priority Date: 03/31/2010
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ogale, Abhijit, Lafon, Stephane
• Application Number:
  US12/752,007
• Publication Number:
  US 20110242271A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

A method for synthesizing three-dimensional images is provided. A displacement map based on distance data, is generated for a two-dimensional image. The distance data represents a distance between an image object and a position. A shifted version of the two-dimensional image is then produced based on the displacement map. The two-dimensional image and the shifted version of the two-dimensional image are then combined to produce a three-dimensional image. A system is also presented.

Claims

1. A method for synthesizing three-dimensional images, comprising:

generating, using a computer-based system, a displacement map, based on distance data, for a first two-dimensional image, wherein the distance data represents a distance between an image object and a first position;

producing a shifted version of the first two-dimensional image based on the displacement map; and

combining the first two-dimensional image and the shifted version of the first two-dimensional image to produce a first three-dimensional image.

2. The method of claim 1, wherein the distance data represents, for each of a plurality of pixels of the first two-dimensional image, the distance from the first position to the image object represented in the first two-dimensional image at each pixel.

3. The method of claim 2, wherein generating the displacement map comprises calculating a displacement value for each of the plurality of pixels, the displacement value being inversely proportional to the distance from the first position to the image object represented in the first two-dimensional image at each pixel.

4. The method of claim 3, wherein generating the displacement map further comprises determining the displacement value for each of the plurality of pixels based on a constant scale factor.

5. The method of claim 3, wherein producing the shifted version of the first two-dimensional image comprises:

laterally shifting each of the plurality of pixels by the displacement value of each of the plurality of pixels.

6. The method of claim 5, wherein combining the first two-dimensional image and the shifted version of the first two-dimensional image comprises:

filtering one or more first color channels of the first two-dimensional image to produce a first component of the first three-dimensional image;

filtering a second color channel, chromatically opposite to the first color channel, of the shifted version of the first two-dimensional image to produce a second component of the first three-dimensional image; and

combining the one or more first color channels of the first component of the first three-dimensional image with the second color channel of the second component of the first three-dimensional image to produce the first three-dimensional image.

7. The method of claim 1, further comprising:

displaying the first three-dimensional image.

8. The method of claim 7, wherein displaying the first three-dimensional image comprises:

displaying a viewport on a portion of the first three-dimensional image, the viewport including a three-dimensional overlay rendered with the first three-dimensional image; and

changing the three-dimensional overlay'"'"'s orientation in three dimensional space as it is rendered with the first three-dimensional image so as to match a change in orientation of the viewport within the first three-dimensional image.

9. The method of claim 1, further comprising:

producing a second three-dimensional image based on a second two-dimensional image; and

combining the first and second three-dimensional images to form a first panoramic three-dimensional image.

10. The method of claim 9, wherein the first panoramic three-dimensional image comprises a 360 degree view.

11. The method of claim 10, further comprising:

producing a second panoramic three-dimensional image based on a second position.

12. The method of claim 1, wherein the displacement map is based on one or more of laser range data and image matching.

13. A system for synthesizing three-dimensional images, comprising:

a displacement map generator to generate a displacement map, based on distance data, for a first two-dimensional image, wherein the distance data represents a distance between an image object and a first position;

an image shifter to produce a shifted version of the first two-dimensional image based on the displacement map; and

an image synthesizer system to combine the first two-dimensional image and the shifted version of the first two-dimensional image to produce a first three-dimensional image.

14. The system of claim 13, wherein the distance data represents, for each of a plurality of pixels of the first two-dimensional image, the distance from the first position to the image object represented in the first two-dimensional image at each pixel.

15. The system of claim 14, wherein the displacement map generator is configured to generate a displacement value for each of the plurality of pixels, the displacement value being inversely proportional to the distance from the first position to the image object represented in the image at each pixel.

16. The system of claim 15, wherein the displacement map generator is configured to determine the displacement value for each of the plurality of pixels based on a constant scale factor.

17. The system of claim 16, wherein the image shifter is configured to laterally shift each of the plurality of pixels by the displacement value of each of the plurality of pixels.

18. The system of claim 17, wherein the image synthesizer system is configured to filter one or more first color channels of the first two-dimensional image to produce a first component of the first three-dimensional image, further configured to filter a second color channel, chromatically opposite to the first color channel, of the shifted version of the first two-dimensional image to produce a second component of the first three-dimensional image, and further configured to combine the one or more first color channels of the first component of the first three-dimensional image with the second color channel of the second component of the first three-dimensional image to produce the first three-dimensional image.

19. The system of claim 13, further comprising:

a rendering device configured to display the first three-dimensional image.

20. The system of claim 19, wherein the rendering device is configured to display a viewport on a portion of the first three-dimensional image, the viewport including a three-dimensional overlay rendered with the first three-dimensional image and further configured to change the three-dimensional overlay'"'"'s orientation in three dimensional space as it is rendered with the first three-dimensional image so as to match a change in orientation of the viewport within the first three-dimensional image.

21. The system of claim 13, further comprising:

a three-dimensional imaging module to combine the first three-dimensional image with a second three-dimensional image to form a first panoramic three-dimensional image.

22. The system of claim 21, wherein the first panoramic three-dimensional image comprises a 360 degree view.

23. The system of claim 21, wherein the three-dimensional imaging module is configured to produce a second panoramic three-dimensional image based on a second position.

24. The system of claim 13, wherein the displacement map is based on one or more of laser range data and image matching.