Alternate Viewpoint Image Enhancement

• US 20160133044A1
• Filed: 01/18/2016
• Published: 05/12/2016
• Est. Priority Date: 06/28/2012
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Lynch, James D.
• Granted Patent: US 10,030,990 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G09B 29/007 using computer methods

Abstract

In one embodiment, panoramic images, images bubbles, or any two-dimensional views of three-dimensional subject matter are enhanced with one or more alternate viewpoints. A controller receives data indicative of a point on the two-dimensional perspective and accesses a three-dimensional location based on the point. The controller selects an image bubble based on the three-dimensional location. The three-dimensional location may be determined according to a depth map corresponding to the point. A portion of the image bubble is extracted and incorporated into the two-dimensional perspective. The resulting image may be a seamless enhanced resolution image or include a picture-in-picture enhanced resolution window including subject matter surrounding the selected point.

Claims

1. (canceled)

2. A method comprising:

receiving first image data for a first image bubble captured from a first perspective;

receiving second image data for a second image bubble captured from a second perspective;

combining, using a processor, the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble,

wherein the composite image is from a third perspective different than the first perspective and different than the second perspective.

3. The method of claim 2, further comprising:

receiving a first depth map associated with the first image bubble;

receiving a second depth map associated with the first image bubble,

wherein the composite image is generated based on the first depth map and the second depth map.

4. The method of claim 2, wherein the first depth map and the second depth map includes depth values for three-dimensional locations in the first and second image bubbles.

5. The method of claim 2, further comprising:

identifying a third image, wherein the first image bubble and the second image bubble are combined with the third image to create the composite image.

6. The method of claim 5, further comprising:

identifying an object of interest in the first image;

identifying the object of interest in the third image, wherein the object of interest in the first image has a higher resolution than the object of interest in the third image.

7. The method of claim 6, wherein the object of interest is identified from an image processing algorithm.

8. The method of claim 7, wherein the image processing algorithm includes edge detection, object recognition, optical character recognition, or feature extraction.

9. The method of claim 2, further comprising:

receiving a user selection of a point of the second image data; and

transitioning from the first perspective to the second perspective in response to the user selection.

10. The method of claim 2, wherein the at least a portion of the first image bubble includes scenery obstructed from the third perspective.

11. The method of claim 2, further comprising:

receiving a command to manipulate the composite image by view angle or zoom.

12. An apparatus comprising:

a memory configured to store first image data for a first image bubble captured from a first perspective and second image data for a second image bubble captured from a second perspective;

a processor configured to combine the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble,

wherein the composite image is from a third perspective different than the first perspective and different than the second perspective.

13. The apparatus of claim 12, wherein the processor is configured to access a first depth map associated with the first image bubble and a second depth map associated with the first image bubble,

wherein the composite image is generated based on the first depth map and the second depth map.

14. The apparatus of claim 12, wherein the first depth map and the second depth map includes depth values for three-dimensional locations in the first and second image bubbles.

15. The apparatus of claim 12, wherein the processor is configured to identify a third image, wherein the first image bubble and the second image bubble are combined with the third image to create the composite image.

16. The apparatus of claim 12, wherein the processor is configured to identify an object of interest in the first image and another view of the object of interest in the third image, wherein the object of interest in the first image has a higher resolution than the object of interest in the third image.

17. The apparatus of claim 16, wherein the processor is configured to receive a user selection of a point of the second image data and transition from the first perspective to the second perspective in response to the user selection.

18. A method comprising:

receiving first image data for a first image bubble captured from a first perspective;

receiving second image data for a second image bubble captured from a second perspective;

receiving a user selection of a point of the second image data; and

transitioning, using a processor, from the first perspective to the second perspective in response to the user selection.

19. The method of claim 18, further comprising:

combining the first image data and the second image data into a composite image, wherein the composite image includes a spatial resolution greater than a resolution of the first image bubble and a resolution of the second image bubble,

wherein the composite image is from a third perspective different than the first perspective and different than the second perspective.

20. The method of claim 18, wherein the at least the portion of the first image bubble includes scenery obstructed from the third perspective.

21. An apparatus comprising:

a memory configured to store first image data for a first image bubble captured from a first perspective and second image data for a second image bubble captured from a second perspective;

an interface configured to receive a user selection of a point of the second image data; and

a processor configured to calculate a transition from the first perspective to the second perspective in response to the user selection.