Stereo panoramic images

• US 9,924,156 B2
• Filed: 11/17/2015
• Issued: 03/20/2018
• Est. Priority Date: 01/31/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Barnes, Craig
• Primary Examiner(s): Harold, Jefferey F
• Assistant Examiner(s): Rahman, Mustafizur
• Application Number:
  US14/943,581
• Publication Number:
  US 20160080725A1
• Time in Patent Office: 854 Days
• Field of Search: 348 36, 348 37, 348 39, 348 43, 348 46, 348 47, 348 48, 348 49, 348 51, 348 52, 348 53, 348264, 348 55, 348 60, 348 77, 348 78, 348119, 348135, 348139, 348140, 348148, 348149, 348290, 348719, 382117, 382118, 382154, 382171, 382173, 382305, 701 23, 701533, 701 26, 701 28, 701400, 701408, 701409, 701410, 701411, 701454, 701460, 701523
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 2013/0077 Colour aspects
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In one embodiment, range data is combined with an image in order to create a stereo image from the single collected image. An apparatus receives image data associated with a geographic location and range data associated with the geographic location. The range data may be a point cloud. At least one perspective image is formed based on the image data and the range data. The perspective image has a viewpoint different than single collected image. The apparatus generates a stereo image from the at least one perspective image. The stereo image appears to have depth. The at least one perspective image may include two images, each offset by a distance to estimate eye separation.

Claims

1. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive a two-dimensional image associated with a geographic location;

receive three-dimensional location data associated with the geographic location that describes distances to objects at the geographic location;

form a left perspective image and a right perspective image from the three-dimensional location data by extraction of color information from the two-dimensional image and insertion of the color information in the three-dimensional location data; and

generate a stereo image formed from the left perspective image and the right perspective image.

2. The apparatus of claim 1, wherein the left perspective image and the right perspective image are offset by an eye separation distance.

3. The apparatus of claim 2, wherein the eye separation distance is in the range of 0 to 2 degrees.

4. The apparatus of claim 1, wherein the stereo image is an anaglyph comprising a first filtered image and a second filtered image.

5. The apparatus of claim 1, wherein the at least one processor is configured to cause the apparatus to calculate a texturing area around a point in the three-dimensional location data and import color information from a plurality of pixels in the two-dimensional image into the texturing area.

6. The apparatus of claim 1, wherein no image data is collected from a viewpoint of the left perspective image or a viewpoint of the right perspective image.

7. The apparatus of claim 1, wherein the left perspective image and the right perspective image comprise multiple image projections.

8. The apparatus of claim 1, further comprising:

a display configured to display the stereo image as part of a navigation application.

9. The apparatus of claim 1, wherein the stereo image is displayed in a map application for accessing street level images along a map.

10. The apparatus of claim 1, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform:

identify a viewpoint oriented quadrilateral from a spherical image bubble, wherein the two-dimensional image is derived from the viewpoint oriented quadrilateral.

11. The apparatus of claim 1, wherein receive the two-dimensional image and the three-dimensional location data are collected at a same time.

12. A method comprising:

receiving a two-dimensional image associated with a geographic location;

receiving three-dimensional location data associated with the geographic location that describes distances to objects at the geographic location;

forming a left perspective image and a right perspective image from the three-dimensional location data by extraction of color information from the two-dimensional image and insertion of the color information in the three-dimensional location data; and

generating a stereo image formed from the left perspective image and the right perspective image.

13. The method of claim 12, wherein the left perspective image and the right perspective image are offset by an eye separation distance.

14. The method of claim 13, wherein the eye separation distance is in the range of 0 to 2 degrees.

15. The method of claim 12, wherein the stereo image is an anaglyph comprising a first filtered image and a second filtered image.

16. The method of claim 15, wherein the first filtered image corresponds to a first color filter and the second filtered image corresponds to a second color filter, wherein the first color filter is chromatically opposite to the second color filter.

17. The method of claim 12, further comprising:

calculating a texturing area around a point in the three-dimensional location data; and

importing color information from a plurality of pixels in the two-dimensional image into the texturing area.

18. The method of claim 12, wherein the stereo image is displayed in a map application for accessing street level images along a map.

19. The method of claim 12, further comprising:

identifying a viewpoint oriented quadrilateral from a spherical image bubble, wherein the two-dimensional image is derived from the viewpoint oriented quadrilateral.

20. A non-transitory computer readable medium including instructions that when executed are operable to:

receive a two-dimensional image associated with a geographic location;

receive three-dimensional location data associated with the geographic location that describes distances to objects depicted in the two-dimensional image;

form a plurality of perspective images from the three-dimensional location data by texturing the three-dimensional location data using color information from the two-dimensional image; and

generate an anaglyph image formed from the plurality of perspective images.