Method and apparatus for an augmented reality user interface

• US 8,427,508 B2
• Filed: 06/25/2009
• Issued: 04/23/2013
• Est. Priority Date: 06/25/2009
• Status: Expired

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Mattila, Jouka Mikael, Nieminen, Hannu Olavi
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US12/491,719
• Publication Number:
  US 20100328344A1
• Time in Patent Office: 1,398 Days
• Field of Search: None
• US Class Current: 345/634
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/027 Arrangements and methods specific for the display of internet documents
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An approach is provided for an augmented reality user interface. An image representing a physical environment is received. Data relating to a horizon within the physical environment is retrieved. A section of the image to overlay location information based on the horizon data is determined. Presenting of the location information within the determined section to a user equipment is initiated.

Claims

1. A method comprising:

receiving an image representing a physical environment;

determining data relating to a horizon within the physical environment;

determining a section of the image to overlay location information with a boundary of the section defined based, at least in part, on the horizon; and

causing, at least in part, a presenting of the location information within the determined section to a user equipment.

2. A method of claim 1, wherein the horizon data comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

3. A method of claim 1, the method further comprising:

receiving the image from an image sensor, and wherein the horizon data is retrieved by processing the image.

4. A method of claim 1, further comprising:

determining structural information relating to the image; and

determining another section of the image comprising structures to overlay the structural information based on the horizon data.

5. A method of claim 1, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

6. A method of claim 1, wherein the location information comprises a map and wherein the map represents a virtual reflection of an area surrounding the physical environment.

7. A method of claim 1, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon.

8. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive an image representing a physical environment,

determine data relating to a horizon within the physical environment,

determine a section of the image to overlay location information with a boundary of the section defined based, at least in part, on the horizon, and

cause, at least in part, a presenting of the location information within the determined section to a user equipment.

9. An apparatus of claim 8, wherein the horizon data comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

10. An apparatus of claim 8, wherein the apparatus is further caused to:

receive the image from an image sensor, and wherein the horizon data is retrieved by processing the image.

11. An apparatus of claim 8, wherein the apparatus is further caused to:

determine structural information relating to the image; and

determine another section of the image comprising structures to overlay the structural information based on the horizon data.

12. An apparatus of claim 8, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

13. An apparatus of claim 8, wherein the location information comprises a map and wherein the map represents a virtual reflection of an area surrounding the physical environment.

14. An apparatus of claim 8, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon.

15. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform at least the following:

receive an image representing a physical environment,

determine data relating to a horizon within the physical environment,

determine a section of the image to overlay location information with a boundary of the section defined based, at least in part, on the horizon, and

cause, at least in part, a presenting of the location information within the determined section to a user equipment.

16. A non-transitory computer-readable storage medium of claim 15, wherein the horizon data comprises data retrieved from an accelerometer sensor, a compass sensor, a gyroscope, an altimeter, or a combination thereof.

17. A non-transitory computer-readable storage medium of claim 15, wherein the apparatus is further caused to:

receive the image from an image sensor, and wherein the horizon data is retrieved by processing the image.

18. A non-transitory computer-readable storage medium of claim 15, wherein the apparatus is further caused to:

determine structural information relating to the image; and

determine another section of the image comprising structures to overlay the structural information based on the horizon data.

19. A non-transitory computer-readable storage medium of claim 15, wherein the location information comprises a map, an icon representing a point of interest, or a combination thereof.

20. A non-transitory computer-readable storage medium of claim 15, wherein the boundary of the section is defined based on outlines of structures within the image above the horizon.