Method and apparatus for an augmented reality user interface

• US RE46,737 E1
• Filed: 04/22/2015
• Issued: 02/27/2018
• Est. Priority Date: 06/25/2009
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Mattila, Jouka Mikael, Nieminen, Hannu Olavi
• Primary Examiner(s): Basehoar, Adam L
• Application Number:
  US14/693,420
• Time in Patent Office: 1,042 Days
• Field of Search: 345629-641, 715764, 715781, 715810, 715823, 715846, 715848-851, 715864
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/027 Arrangements and methods specific for the display of internet documents
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/04 using association of physical positions and logical data in a dedicated environment, e.g. buildings or vehicles

Abstract

An approach is provided for an augmented reality user interface. An image representing a physical environment is received. Data relating to a horizon within the physical environment is retrieved. A section of the image to overlay location information based on the horizon data is determined. Presenting of the location information within the determined section to a user equipment is initiated.

Claims

1. A method comprising:

receiving an image representing a physical environment;

determiningretrieving data relating to a horizon within the physical environment;

determining a first section of the image to overlay in which to present location information with a boundary of the section defined based, at least in part, on the horizon; and

based on the data relating to the horizon within the physical environment;

causing, at least in part, a presenting presentation of the location information within the determined first section to a user equipment of the image, wherein the location information comprises a map and further wherein the map represents a virtual reflection of an area surrounding the physical environment; and

determining a virtual horizon line based on virtual objects in a virtual world corresponding to the physical environment,

wherein presentation of the virtual reflection of the map is a virtual dome representation of the map of the area surrounding the physical environment.

2. A method of claim 1, wherein the horizon data relating to the horizon within the physical environment comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

3. A method of claim 1, the method further comprising:

receiving the image from an image sensor, and wherein the horizon data relating to the horizon within the physical environment is retrieved by processing the image.

4. A method of claim 1, further comprising:

determiningretrieving structural information relating to the image; and

determining another a second section of the image comprising that comprises structures to overlay in which to present the structural information based on the horizon data relating to the horizon within the physical environment.

5. A method of claim 1, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

6. A method of claim 1, wherein the location information comprises a map and wherein the map represents a virtual reflection of an area surrounding the physical environment.

7. A method of claim 1, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon

wherein the section of the image in which to present the location information is above the virtual horizon.

8. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive an image representing a physical environment,

determineretrieve data relating to a horizon within the physical environment,

determine a first section of the image to overlay in which to present location information with a boundary of the section defined based, at least in part, on the horizon, and based on the data relating to the horizon within the physical environment,

cause, at least in part, a presenting presentation of the location information within only the determined first section to a user equipment of the image, wherein the location information comprises a map and further wherein the map represents a virtual reflection of an area surrounding the physical environment, and

determining a virtual horizon line based on virtual objects in a virtual world corresponding to the physical environment,

wherein presentation of the virtual reflection of the map is a virtual dome representation of the map of the area surrounding the physical environment.

9. An apparatus of claim 8, wherein the horizon data relating to the horizon within the physical environment comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

10. An apparatus of claim 8, wherein the apparatus is further caused to:

receive the image from an image sensor, and wherein the horizon data relating to the horizon within the physical environment is retrieved by processing the image.

11. An apparatus of claim 8, wherein the apparatus is further caused to:

determineretrieve structural information relating to the image; and

determine another a second section of the image comprising that comprises structures to overlay in which to present the structural information based on the horizon data relating to the horizon within the physical environment.

12. An apparatus of claim 8, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

13. An apparatus of claim 8, wherein the location information comprises a map and wherein the map represents a virtual reflection of an area surrounding the physical environment.

14. An apparatus of claim 8, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon

wherein the section of the image in which to present the location information is above the virtual horizon.

15. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform at least the following:

receive an image representing a physical environment,

determineretrieve data relating to a horizon within the physical environment,

determine a first section of the image to overlay in which to present location information with a boundary of the section defined based, at least in part, on the horizon, and based on the data relating to the horizon within the physical environment,

cause, at least in part, a presenting presentation of the location information within the determined first section to a user equipment of the image, wherein the location information comprises a map and further wherein the map represents a virtual reflection of an area surrounding the physical environment, and

determining a virtual horizon line based on virtual objects in a virtual world corresponding to the physical environment,

wherein presentation of the virtual reflection of the map is a virtual dome representation of the map of the area surrounding the physical environment.

16. A non-transitory computer-readable storage medium of claim 15, wherein the horizon data relating to the horizon within the physical environment comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

17. A non-transitory computer-readable storage medium of claim 15, wherein the apparatus is further caused to:

receive the image from an image sensor, and wherein the horizon data relating to the horizon within the physical environment is retrieved by processing the image.

18. A non-transitory computer-readable storage medium of claim 15, wherein the apparatus is further caused to:

determineretrieve structural information relating to the image; and

determine another a second section of the image comprising that comprises structures to overlay in which to present the structural information based on the horizon data relating to the horizon within the physical environment.

19. A non-transitory computer-readable storage medium of claim 15, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

20. A non-transitory computer-readable storage medium of claim 15, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon

wherein the section of the image in which to present the location information is above the virtual horizon.