Method and apparatus for correlating and navigating between a live image and a prerecorded panoramic image

• US 9,372,094 B2
• Filed: 07/08/2011
• Issued: 06/21/2016
• Est. Priority Date: 12/14/2009
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Kankainen, Mikko Tapio
• Primary Examiner(s): WANG, JIN CHENG
• Application Number:
  US13/179,089
• Publication Number:
  US 20120033032A1
• Time in Patent Office: 1,810 Days
• Field of Search: 382/284, 348/14.01, 348/143, 348/218.1, 348/E5.0583, 345/629
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

An approach is provided for correlating and navigating between a live camera image and a prerecorded panoramic image. A mapping and augmented reality application correlates at least one live image with a prerecorded panoramic image, when a first location of a device used to capture the at least one live image substantially matches or falls within a predetermined proximity of a second location of a device used to capture the panoramic prerecorded image. The mapping and augmented reality application causes, at least in part, alternating of the at least one live image and the prerecorded panoramic image in a presentation on a screen of the device capturing the at least one live image.

Claims

1. A method comprising:

determining, by an apparatus, a correlation of at least one live image with a prerecorded panoramic image, when a first location of a mobile device embedded with the apparatus and used to capture the at least one live image substantially matches a second location of another device used to capture the panoramic prerecorded image;

in response to a user input, causing, by the apparatus, an augment of one or more indicators into the at least one live image; and

in response to a user touch input at an edge of the at least one live image fully occupying a screen of the device capturing the at least one live image, causing, by the apparatus, an alternating of the at least one live image with the prerecorded panoramic image fully occupying the screen of the device, wherein a view shown in the prerecorded panoramic image continues at the edge with a view shown in the at least one live image without overlapping.

2. A method of claim 1, wherein the at least one live image is a live camera view seen via the screen of the device.

3. A method of claim 2, further comprising:

when the user touch input continues touching the edge of the screen of the device, causing other prerecorded panoramic images containing continuous views outside of the view shown in the prerecorded panoramic image to roll out and fully occupy the screen.

4. A method of claim 2, further comprising:

in response to a user touch input at a point of the at least one live image shown on the screen of the device, causing an alternating of the at least one live image into a prerecorded panoramic image taken from a location corresponding to the point of the at least one live image.

5. A method of claim 4, further comprising:

in response to a rotation, a tilt, or a combination thereof, of the device, causing an alternating of the prerecorded panoramic image taken from the location to one or more other prerecorded panoramic images taken from the location based, at least in part, on the rotation, the tilt, or a combination thereof.

6. A method of claim 4, further comprising:

in response to a rotation, a tilt, or a combination thereof, of the device towards another location, determining a point in the prerecorded panoramic image corresponding to the another location; and

causing an augment of a human-shaped avatar representing a user of the device into the prerecorded panoramic image.

7. A method of claim 1, wherein the at least one live image shows a commercial establishment, the one or more indicators include market information, and the method further comprising:

in response to one or more updates to the market information, causing an augment of the one or more updates into the at least one live image on real time.

8. A method of claim 7, wherein the at least one live image shows an indoor view of the commercial establishment.

9. A method of claim 1, wherein the one or more indicators are augmented into the at least one live image in a real time manner with the user input.

10. A method of claim 1, wherein the one or more indicators include a human-shaped avatar representing a user of the device capturing the at least one live image.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

determine a correlation of at least one live image with a prerecorded panoramic image, when a first location of a mobile device embedded with the apparatus and used to capture the at least one live image substantially matches a second location of another device used to capture the panoramic prerecorded image;

in response to a user input, cause an augment of one or more indicators into the at least one live image; and

in response to a user touch input at an edge of the at least one live image fully occupying a screen of the device capturing the at least one live image, cause an alternating of the at least one live image with the prerecorded panoramic image fully occupying the screen of the device, wherein a view shown in the prerecorded panoramic image continues at the edge with a view shown in the at least one live image without overlapping.

12. An apparatus of claim 11, wherein the at least one live image is a live camera view seen via the screen of the device.

13. An apparatus of claim 12, wherein the apparatus is further caused to:

when the user touch input continues touching the edge of the screen of the device, cause other prerecorded panoramic images containing continuous views outside of the view shown in the prerecorded panoramic image to roll out and fully occupy the screen.

14. An apparatus of claim 12, wherein the apparatus is further caused to:

in response to a user touch input at a point of the at least one live image shown on the screen of the device, cause an alternating of the at least one live image into a prerecorded panoramic image taken from a location corresponding to the point of the at least one live image.

15. An apparatus of claim 14, wherein the apparatus is further caused to:

in response to a rotation, a tilt, or a combination thereof, of the device, cause an alternating of the prerecorded panoramic image taken from the location to one or more other prerecorded panoramic images taken from the location based on the rotation, the tilt, or a combination thereof.

16. An apparatus of claim 14, wherein the apparatus is further caused to:

in response to a rotation, a tilt, or a combination thereof, of the device towards another location, determine a point in the prerecorded panoramic image corresponding to the another location; and

cause an augment of a human-shaped avatar representing a user of the device into the prerecorded panoramic image.

17. An apparatus of claim 11, wherein the at least one live image shows a commercial establishment, the one or more indicators include market information, and the method further comprising:

in response to one or more updates to the market information, cause an augment of the one or more updates into the at least one live image on real time.

18. An apparatus of claim 17, wherein the at least one live image shows an indoor view of the commercial establishment.

19. An apparatus of claim 11, wherein the one or more indicators are augmented into the at least one live image in a real time manner with the user input.

20. An apparatus of claim 11, wherein the one or more indicators include a human-shaped avatar representing a user of the device capturing the at least one live image.