PROJECTING LOCATION BASED ELEMENTS OVER A HEADS UP DISPLAY

• US 20110052042A1
• Filed: 06/01/2010
• Published: 03/03/2011
• Est. Priority Date: 08/26/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Mishor 3D LTD
• Inventors: BEN TZVI, Jacob
• Granted Patent: US 8,503,762 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

Methods and systems for projecting a location based elements over a heads up display. One method includes: generating a three dimensional (3D) model of a scene, based on a source of digital mapping of the scene; associating a position of at least one selected LAE contained within the scene, with a respective position in the 3D model; superimposing the projecting onto a specified position on a transparent screen facing a viewer and associated with the vehicle, at least one graphic indicator associated with the at least one LAE, wherein the specified position is calculated based on: the respective position of the LAE in the 3D model, the screen'"'"'s geometrical and optical properties, the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, the vehicle'"'"'s position and angle within the scene, such that the viewer, the graphic indicator, and the LAE are substantially on a common line.

Claims

1. A method comprising:

generating a three dimensional (3D) model of a scene within a specified radius from a vehicle, based on a digital mapping source of the scene;

associating a position of at least one selected location aware entity (LAE) contained within the scene, with a respective position in the 3D model; and

superimposing the LAE from the 3D model onto a screen in a specified position and in a form of a graphic indicator facing a viewer and associated with the vehicle,

wherein the specified position is calculated at least partially based on at least one of;

the respective position of the LAE in the 3D model, the screen'"'"'s geometrical and optical properties, the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, the vehicle'"'"'s position and angle within the scenery, such that the viewer, the graphic indicator, and the LAE are substantially on a common line.

2. The method according to claim 1, wherein the graphic indicator is a symbolic representation of the LAE.

3. The method according to claim 1, wherein the graphic indicator is a direction arrow useable for directing the viewer to the LAE, and wherein the position of the direction arrow on the screen and the direction arrow'"'"'s graphical representation is calculated based on at least one of:

location, angle, and direction of a turning point within the scene in association with which a turn has to be made in order to reach the LAE.

4. The method according to claim 3, wherein the direction arrow'"'"'s direction is repeatedly updated based on the vehicle'"'"'s current position and on a calculated route between the vehicle and the LAE.

5. The method according to claim 1, further comprising updating at least one of:

the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, and the vehicle'"'"'s position and angle within the scene, to yield an updated 3D model and updated position of the at least one graphic indicator.

6. The method according to claim 1, wherein in case a specified LAE is obscured by an element contained within the scene and closer to the viewer, the position of the graphic indicator associated with the specified LAE is adjusted to a specified distance from a LAE on the screen substantially on a common line with the viewer and the specified LAE.

7. The method according to claim 1, further comprising:

determining a specified area of display based on at least;

display spatial angle, vehicle dimensions; and

calibrating the determined area based on viewer'"'"'s angle to yield a dynamically updated area of display projected inside of a specified matrix usable for excluding the range &

orientation to the LAE and also the spatial fixed.

8. A system comprising:

a processor; and

a projector configured to;

generate a three dimensional (3D) model of a scene within a specified radius from a vehicle, based on a digital mapping source of the scene and associate a position of at least one selected LAE contained within the scene, with a respective position in the 3D model; and

superimpose the selected LAE from the 3D model onto a 2D specified projection area,

wherein the projector is configured to project onto a specified position on a transparent screen facing a viewer and associated with the vehicle, at least one graphic indicator associated with the superimposed LAE, and wherein the specified position is calculated by the processor based on;

the respective position of the LAE in the 3D model, the screen'"'"'s geometrical and optical properties, the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, the vehicle'"'"'s position and angle within the scene, such that the viewer, the graphic indicator, and the LAE are substantially on a common line.

9. The system according to claim 8, wherein the graphic indicator is a symbolic representation of the LAE.

10. The system according to claim 8, wherein the graphic indicator is a direction arrow useable for directing the viewer to the LAE, and wherein the position of the direction arrow on the screen is calculated based on at least one of:

location, angle, and direction of a turning point within the scene in association with which a turn has to be made in order to reach the LAE.

11. The system according to claim 10, wherein the direction arrow'"'"'s direction is repeatedly updated based on the vehicle'"'"'s current position and on a calculated route between the vehicle and the LAE.

12. The system according to claim 8, wherein the processor is further configured to update at least one of:

the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, and the vehicle'"'"'s position and angle within the scene, to yield an updated 3D model and updated position of the at least one graphic indicator.

13. The system according to claim 12, wherein at least some of the update is estimated by the processor.

14. The system according to claim 8, wherein, when a specified LAE is obscured by an element contained within the scene and closer to the viewer, the position of the graphic indicator associated with the specified LAE is adjusted by the processor to a specified distance from a LAE on the screen substantially on a common line with the viewer and the specified LAE.

15. A computer program product, the computer program product comprising:

a computer readable storage medium having a computer readable program embodied therewith, the computer readable program comprising;

computer readable code configured to generate a three dimensional (3D) model of a scene within a specified radius from a vehicle, based on a digital mapping source of the scene;

computer readable code configured to associate a position of at least one selected LAE contained within the scene, with a respective position in the 3D model; and

computer readable code configured to calculate a position of at least one graphic indicator associated with at least one LAE and projected onto a specified position on a transparent screen facing a viewer and associated with the vehicle, wherein the calculation is based on;

the respective position of the LAE in the 3D model, the screen'"'"'s geometrical and optical properties, the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, the vehicle'"'"'s position and angle within the scene, such that the viewer, the graphic indicator, and the LAE are substantially on a common line.

16. The computer program product according to claim 15, wherein the graphic indicator is a symbolic representation of the LAE.

17. The computer program product according to claim 15, wherein the graphic indicator is a direction arrow useable for directing the viewer to the LAE, and wherein the position of the direction arrow on the screen is calculated based on at least one of:

location, angle, and direction of a turning point within the scene in association with which a turn has to be made in order to reach the LAE.

18. The computer program product according to claim 17, wherein the direction arrow'"'"'s (DDE) direction is repeatedly updated based on the vehicle'"'"'s current position and on a calculated route between the vehicle and the LAE.

19. The computer program product according to claim 15, further comprising computer readable code configured to update at least one of:

the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, and the vehicle'"'"'s position and angle within the scene, to yield an updated 3D model and updated position of the at least one graphic indicator.