Method and apparatus for providing a localized virtual reality environment

• US 9,122,707 B2
• Filed: 05/28/2010
• Issued: 09/01/2015
• Est. Priority Date: 05/28/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Wither, Jason Robert, Azuma, Ronald
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US12/790,288
• Publication Number:
  US 20110292076A1
• Time in Patent Office: 1,922 Days
• Field of Search: 345/632
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 2216/13 Prefetching
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An apparatus for enabling provision of a localized virtual reality environment may include at least one processor and at least one memory including computer program code. The at least one memory and the computer program code may be configured, with the processor, to cause the apparatus to perform at least receiving information indicative of a current location of a mobile terminal, receiving information indicative of an orientation of the mobile terminal with respect to the current location, causing a stored image including a panoramic view of the current location to be displayed at the mobile terminal based on the orientation, and enabling provision of a virtual object on the panoramic view. A corresponding method and computer program product are also provided.

Claims

1. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least:

receive information indicative of a current location of a mobile terminal;

receive information indicative of an orientation of the mobile terminal with respect to the current location;

select a stored image including a panoramic view of the current location from among a plurality of images of the current location having different lighting conditions, wherein the stored image is selected based upon a current time of day or current weather condition such that the lighting condition of the stored image corresponds to the current time of day or the current weather condition;

cause the stored image including the panoramic view of the current location to be displayed at the mobile terminal based on the orientation; and

enable provision of a virtual object on the panoramic view by associating the virtual object with a corresponding real object within the panoramic view.

2. The apparatus of claim 1, wherein the memory and computer program code are configured to, with the processor, cause the apparatus to cause the stored image to be displayed by further selecting the stored image from among a plurality of previously captured images, each of the previously captured images being associated with a corresponding location and orientation.

3. The apparatus of claim 1, wherein the memory and computer program code are configured to, with the processor, cause the apparatus to cause the stored image to be displayed by further selecting the stored image from among a plurality of previously captured images pre-fetched from a remote device and locally stored, the images pre-fetched being selected based on a proximity of each pre-fetched image relative to the current location.

4. The apparatus of claim 1, wherein the memory and computer program code are configured to, with the processor, cause the apparatus to enable provision of the virtual object by replacing a real object with the virtual object or removing the real object.

5. The apparatus of claim 1, wherein the memory and computer program code are configured to, with the processor, cause the apparatus to receive information indicative of the orientation of the mobile terminal by comparing a reference point in a live image to a prior position of the reference point in a previously captured image or frame.

6. The apparatus of claim 1, wherein the apparatus is remotely located from the mobile terminal such that receiving information indicative of the current location of the mobile terminal and receiving information indicative of the orientation of the mobile terminal occurs at a device remote with respect to the mobile terminal and wherein causing the stored image to be displayed comprises providing the stored image from the device to the mobile terminal.

7. The apparatus of claim 1, wherein the apparatus is a mobile terminal and further comprises user interface circuitry configured to facilitate user control of at least some functions of the mobile terminal.

18. The apparatus of claim 1, wherein the memory and computer program code are further configured to, with the processor, cause the apparatus to adjust a scale of the virtual object at least partially in response to a determination that the mobile terminal has moved.

8. A method comprising:

receiving information indicative of a current location of a mobile terminal;

receiving information indicative of an orientation of the mobile terminal with respect to the current location;

selecting a stored image including a panoramic view of the current location from among a plurality of images of the current location having different lighting conditions, wherein the stored image is selected based upon a current time of day or current weather condition such that the lighting condition of the stored image corresponds to the current time of day or the current weather condition;

causing the stored image including the panoramic view of the current location to be displayed at the mobile terminal based on the orientation; and

enabling provision of a virtual object on the panoramic view, wherein enabling provision of the virtual object further comprises associating the virtual object with a corresponding real object within the panoramic view.

9. The method of claim 8, wherein causing the stored image to be displayed comprises selecting the stored image from among a plurality of previously captured images, each of the previously captured images being associated with a corresponding location and orientation.

10. The method of claim 8, wherein causing the stored image to be displayed comprises selecting the stored image from among a plurality of previously captured images pre-fetched from a remote device and locally stored, the images pre-fetched being selected based on a proximity of each pre-fetched image relative to the current location.

11. The method of claim 8, wherein enabling provision of the virtual object comprises replacing a real object with the virtual object or removing the real object.

12. The method of claim 8, wherein receiving information indicative of the orientation of the mobile terminal comprises comparing a reference point in a live image to a prior position of the reference point in a previously captured image or frame.

13. The method of claim 8, wherein receiving information indicative of the current location of the mobile terminal and receiving information indicative of the orientation of the mobile terminal occurs at a device remote with respect to the mobile terminal and wherein causing the stored image to be displayed comprises providing the stored image from the device to the mobile terminal.

19. The method of claim 8, further comprising adjusting a scale of the virtual object at least partially in response to a determination that the mobile terminal has moved.

14. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instruction comprising program code instructions for:

receiving information indicative of a current location of a mobile terminal;

receiving information indicative of an orientation of the mobile terminal with respect to the current location;

selecting a stored image including a panoramic view of the current location from among a plurality of images of the current location having different lighting conditions, wherein the stored image is selected based upon a current time of day or current weather condition such that the lighting condition of the stored image corresponds to the current time of day or the current weather condition;

causing the stored image including the panoramic view of the current location to be displayed at the mobile terminal based on the orientation; and

enabling provision of a virtual object on the panoramic view, wherein program code instructions for enabling provision of the virtual object further include instructions for associating the virtual object with a corresponding real object within the panoramic view.

15. The computer program product of claim 14, wherein program code instructions for causing the stored image to be displayed include instructions for selecting the stored image from among a plurality of previously captured images, each of the previously captured images being associated with a corresponding location and orientation.

16. The computer program product of claim 14, wherein program code instructions for causing the stored image to be displayed include instructions for selecting the stored image from among a plurality of previously captured images pre-fetched from a remote device and locally stored, the images pre-fetched being selected based on a proximity of each pre-fetched image relative to the current location.

17. The computer program product of claim 14, wherein program code instructions for enabling provision of the virtual object include instructions for replacing a real object with the virtual object or removing the real object.

20. The computer program product of claim 14, wherein the computer-executable program code instruction further comprise program code instructions for adjusting a scale of the virtual object at least partially in response to a determination that the mobile terminal has moved.