Method and apparatus for rendering a perspective view of objects and content related thereto for location-based services on mobile device

• US 9,916,673 B2
• Filed: 10/17/2016
• Issued: 03/13/2018
• Est. Priority Date: 05/16/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Castro, Brenda, Vaittinen, Tuomas, Murphy, David Joseph
• Primary Examiner(s): McCulley, Ryan D
• Application Number:
  US15/295,749
• Publication Number:
  US 20170039695A1
• Time in Patent Office: 512 Days
• Field of Search: None
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A method including causing, at least in part, rendering of a perspective view showing one or more objects in a field of view. The method further including retrieving content associated with an object of the one or more objects in the field of view, and causing, at least in part, rendering of a graphic representation relating to the content on a surface of the object visible in the perspective view in a user interface for a location-based service of a mobile device.

Claims

1. A method comprising:

rendering a perspective view showing one or more objects in a field of view;

rendering a plan view of a map showing at least the one or more objects in the field of view, wherein the perspective and plan views are rendered simultaneously;

retrieving content associated with an object of the one or more objects in the field of view;

rendering a graphic representation relating to the content on a surface of the object visible in the perspective view in a user interface for a location-based service of a mobile device;

determining a perspective of the rendering of the perspective view in relation to the one or more objects;

determining whether a rendering of a distant object is obstructed by a rendering of another object in the perspective view;

omitting a graphic representation relating to content associated with the distant object from the perspective view when the distant object is obstructed by the rendering of another object in the perspective view;

determining an optimal image of the one or more objects for the perspective view from real-time images and pre-stored images; and

rendering of the perspective view and the plan view of the map simultaneously in the user interface using the optimal image,

wherein the one or more objects in the perspective view corresponds to the one or more objects in the plan view of the map, and the perspective view and the plan view of the map are switchable while simultaneously rendered in the user interface.

2. The method of claim 1, wherein the rendering of the graphic representation includes orienting the graphic representation with respect to the surface of the object visible in the perspective view and the distant object and the another object are points of interests.

3. The method of claim 1, further comprising:

receiving an input for selecting the graphic representation via the user interface; and

displaying of the content associated with the object on the user interface.

4. The method of claim 1, further comprising:

receiving real-time images captured by the mobile device of the one or more objects in the field of view,

wherein the perspective view shows the real-time images with the graphic representation overlaid onto the surface of the object of the one or more objects in the field of view.

5. The method of claim 1, further comprising:

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view shows the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

6. The method of claim 1, further comprising:

receiving real-time images captured by the mobile device of the one or more objects in the field of view; and

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view shows a mix of the real-time images and the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

7. The method of claim 1, further comprising:

receiving real-time images captured by the mobile device of the one or more objects in the field of view; and

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view transitions between the real-time images, the pre-stored images, and a mix of the real-time images and the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

8. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following;

rendering a perspective view showing one or more objects in a field of view;

rendering a plan view of a map showing at least the one or more objects in the field of view, wherein the perspective and plan views are rendered simultaneously;

retrieving content associated with an object of the one or more objects in the field of view;

rendering a graphic representation relating to the content on a surface of the object visible in the perspective view in a user interface for a location-based service of a mobile device;

determining a perspective of the rendering of the perspective view in relation to the one or more objects;

determining whether a rendering of a distant object is obstructed by a rendering of another object in the perspective view;

omitting a graphic representation relating to content associated with the distant object from the perspective view when the distant object is obstructed by the rendering of another object in the perspective view;

determining an optimal image of the one or more objects for the perspective view from real-time images and pre-stored images; and

rendering of the perspective view and the plan view of the map simultaneously in the user interface using the optimal image,

wherein the one or more objects in the perspective view corresponds to the one or more objects in the plan view of the map, and the perspective view and the plan view of the map are switchable while simultaneously rendered in the user interface.

9. The apparatus of claim 8, wherein the rendering of the graphic representation includes orienting the graphic representation with respect to the surface of the object visible in the perspective view and the distant object and the another object are points of interests.

10. The method of claim 8, wherein the apparatus is further caused to:

receiving an input for selecting the graphic representation via the user interface; and

displaying of the content associated with the object on the user interface.

11. The method of claim 8, wherein the apparatus is further caused to:

receiving real-time images captured by the mobile device of the one or more objects in the field of view,

wherein the perspective view shows the real-time images with the graphic representation overlaid onto the surface of the object of the one or more objects in the field of view.

12. The method of claim 8, wherein the apparatus is further caused to:

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view shows the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

13. The method of claim 8, wherein the apparatus is further caused to:

receiving real-time images captured by the mobile device of the one or more objects in the field of view; and

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view shows a mix of the real-time images and the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

14. The method of claim 8, wherein the apparatus is further caused to:

receiving real-time images captured by the mobile device of the one or more objects in the field of view; and

receiving pre-stored images of the one or more objects in the field of view,

wherein the perspective view transitions between the real-time images, the pre-stored images, and a mix of the real-time images and the pre-stored images with the graphic representation overlaid onto the surface of an object of the one or more objects in the field of view.

15. A non-transitory computer program product including one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the steps:

rendering a perspective view showing one or more objects in a field of view;

rendering a plan view of a map showing at least the one or more objects in the field of view, wherein the perspective and plan views are rendered simultaneously;

retrieving content associated with an object of the one or more objects in the field of view;

rendering a graphic representation relating to the content on a surface of the object visible in the perspective view in a user interface for a location-based service of a mobile device;

determining a perspective of the rendering of the perspective view in relation to the one or more objects;

determining whether a rendering of a distant object is obstructed by a rendering of another object in the perspective view;

omitting a graphic representation relating to content associated with the distant object from the perspective view when the distant object is obstructed by the rendering of another object in the perspective view;

determining an optimal image of the one or more objects for the perspective view from real-time images and pre-stored images; and

rendering of the perspective view and the plan view of the map simultaneously in the user interface using the optimal image,

wherein the one or more objects in the perspective view corresponds to the one or more objects in the plan view of the map, and the perspective view and the plan view of the map are switchable while simultaneously rendered in the user interface.

16. The non-transitory computer program product of claim 15, wherein the rendering of the graphic representation includes orienting the graphic representation with respect to the surface of the object visible in the perspective view and the distant object and the another object are points of interests.

17. The non-transitory computer program product of claim 15, wherein the apparatus is caused, at least in part, to further perform:

receiving an input for selecting the graphic representation via the user interface; and

displaying of the content associated with the object on the user interface.