Image enhancement using a multi-dimensional model

• US 9,454,848 B2
• Filed: 05/20/2013
• Issued: 09/27/2016
• Est. Priority Date: 05/20/2013
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Mattila, Ville-Veikko
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Wang, Yi
• Application Number:
  US13/897,571
• Publication Number:
  US 20140340394A1
• Time in Patent Office: 1,226 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/344 involving models

Abstract

A method and apparatus for enhancing an image capture of a geographic area is presented. The enhancement involves receiving an image and aligning a three dimensional model to the received image. A view of the three dimensional model defined by the outline of the received image may be determined. A point of interest in the determined view of the aligned three dimensional model may be identified. A transition may be triggered between the received image to information from the three dimensional model associated with the point of interest.

Claims

1. A method of image enhancement, the method comprising:

receiving an image captured by a camera, wherein the received image comprises registration information;

displaying the image;

aligning, with a processor, a three dimensional model to the received image, wherein the aligning comprises using the registration information to correlate a point in the model to an actual location where the camera was located when the image was captured;

identifying a plurality of points of interest in the received image based on correlating the points of interest from the aligned three dimensional model to the received image;

transitioning from displaying the received image to displaying textual information regarding one of the points of interest from the three dimensional model associated with the one of the points of interest; and

transitioning between displaying each of the plurality of points of interest from the image in an order based on a point of interest hierarchy.

2. The method of claim 1, wherein the transitioning is triggered automatically after a period of time.

3. The method of claim 1, further comprising:

modifying the received image to include an area of interest that correlates to the point of interest.

4. The method of claim 1, wherein the textual information regarding the point of interest from the three dimensional model comprises a name of the point of interest, hours of operation of the point of interest, or background information relating to the point of interest.

5. The method of claim 1, wherein the correlating is determined by matching geometry between the received image and the three dimensional model.

6. The method of claim 1, wherein the points of interest are identified as existing in a two dimensional view of the three dimensional model, and existence of the points of interest in the two dimensional view implicate existence of the points of interest in the received image.

7. The method of claim 1, wherein the received image comprises geographic data indicating the location of a device that captured the received image, wherein the points of interest are identified using the geographic data.

8. The method of claim 1, wherein the transitioning comprises transitioning away from prominently displaying the received image to prominently displaying the textual information regarding one of the points of interest from the three dimensional model associated with the corresponding point of interest in a viewing area.

9. The method of claim 8, wherein after the transitioning, the textual information fills a majority of the viewing area.

10. An apparatus comprising:

a memory configured to store a captured image and a three dimensional model of a geographic area represented by the image;

position circuitry configured to determine position information; and

a controller configured to;

align the three dimensional model to the image, wherein the geographic area comprises a point of interest, wherein the controller is further configured to associate the position information with the image as registration information, and the registration information is used to align the three dimensional model to the image, and

transition away from displaying the image to displaying textual information regarding the point of interest from the three dimensional model, wherein there are a plurality of points of interest in the geographic area, and the controller is further configured to transition between displaying each of the plurality of points of interest from the image in an order based on a point of interest hierarchy.

11. The apparatus of claim 10, further comprising:

an image capture device configured to capture and generate the image from a physical view of the geographic area.

12. The apparatus of claim 10, wherein the controller is further configured to:

modify the image to include an area of interest that correlates to the point of interest.

13. The apparatus of claim 10, wherein the transitioning is triggered by selecting the area of interest.

14. The apparatus of claim 10, further comprising:

a communication interface configured to download at least a portion of the three dimensional model corresponding to the geographic area.

15. A non-transitory computer readable medium including instructions that when executed by a processor are operable to:

receive an image captured by a camera of a mobile device, wherein the received image comprises registration information;

align the received image to a model, wherein the aligning comprises using the registration information to correlate a point in the model to an actual location where the camera was located when the image was captured;

display the received image in an active view;

identify, based on the model, a plurality of points of interest in the received image that correlate to the plurality of points of interest as represented in the model; and

transition the active view from the received image between the plurality of points of interest and textual information regarding the points of interest from the three dimensional model in an order based on a point of interest hierarchy.

16. The non-transitory computer readable medium of claim 15, wherein the transitioning is automatic.

17. The non-transitory computer readable medium of claim 15, wherein the non-transitory computer readable medium is a memory of a mobile device.

18. The non-transitory computer readable medium of claim 17, wherein the instructions are further operable to cause the processor to:

capture the received image as two dimensional visual data representing a physical view of a geographic area.