Projecting location based elements over a heads up display

• US 9,171,214 B2
• Filed: 08/05/2013
• Issued: 10/27/2015
• Est. Priority Date: 08/26/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Jacob Ben Tzvi
• Inventors: Ben Tzvi, Jacob
• Primary Examiner(s): Harandi, Siamak
• Application Number:
  US13/958,717
• Publication Number:
  US 20130315446A1
• Time in Patent Office: 813 Days
• Field of Search: 382/154
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A method including the following steps is provided: generating a three dimensional (3D) model of a scene within a specified radius from a vehicle, based on a source of digital mapping of the scene; associating a position of at least one selected LAE contained within the scene, with a respective position in the 3D model; superimposing the projecting onto a specified position on a transparent screen facing a viewer and associated with the vehicle, at least one graphic indicator associated with the at least one LAE, wherein the specified position is calculated based on: the respective position of the LAE in the 3D model, the screen'"'"'s geometrical and optical properties, the viewer'"'"'s viewing angle, the viewer'"'"'s distance from the screen, the vehicle'"'"'s position and angle within the scene, such that the viewer, the graphic indicator, and the LAE are substantially on a common line.

Claims

1. A computerized method, comprising:

estimating a current location of a viewer in a vehicle;

estimating a current view direction of a viewer in said vehicle;

identifying in digital surface model, using a processor, a plurality of specified positions of a plurality of points of interest selected according to said estimated current view direction and said current location; and

superimposing a plurality of location graphic symbolic indicators on a see through substance, each one of said plurality of location graphic symbolic indicators is superimposed according to data from a digital surface model to appear over a section of a building viewed via said see through substance in one of said plurality of specified positions;

wherein each one of said plurality of location graphic symbolic indicators is sized and shaped and presented to appear over a section of said building according to said data from the digital surface model and a distance between said current location and a respective of said plurality of specified positions and a direction from said current location to said respective point of interest.

2. The computerized method of claim 1, further comprising:

detecting a new view direction of said viewer;

identifying a plurality of new points of interest selected according to said estimated new view direction; and

superimposing on said see through substance a plurality of new location graphic symbolic indicators instead of said plurality of new location graphic symbolic indicators, each respectively represents a specified position of one of said plurality of new points of interest, said plurality of location graphic symbolic indicators in a plurality of new specified positions selected according to said new estimated current view direction.

3. The computerized method of claim 1, wherein said plurality of location graphic symbolic indicators are projected on a windshield.

4. The computerized method of claim 3, wherein superimposing is performed at a lower section of said windshield.

5. The computerized method of claim 1, wherein each one of said plurality of points of interest is indicative of a member selected from:

a Gas station, a Bank, a Police station, an entertainment event, and a Trade/Shopping center.

6. The computerized method of claim 1, wherein at least one of said plurality of specified positions is calculated to appear above or on a building seen via a see through surface.

7. The computerized method of claim 6, wherein a height of said building is documented in a matrix Structure having length, width, and height dimensions said matrix stores dedicated information about a scene in a section visible.

8. The computerized method of claim 6, wherein said plurality of specified positions are calculated relatively to an orientation of said vehicle.

9. The computerized method of claim 1, wherein at least one of elevation and slopes data of at least one of said plurality of specified positions is identified according to a digital terrain model (DTM) of an area selected according to said current location.

10. The computerized method of claim 9, wherein said superimposing comprises detecting an occlusion of a line of sight of said viewer to at least one of said plurality of points of interest by an occluding object using said DTM and calculating said at least one specified position to appear above said occluding object.

11. The computerized method of claim 9, wherein said superimposing comprises detecting an occlusion of a line of sight of said viewer to at least one of said plurality of points of interest by an occluding object using said DTM and calculating an unblocked view of said at least one point of interest and calculating said at least one specified position to appear on said unblocked view.

12. The computerized method of claim 1, further comprising:

receiving a query defining a desired distance and a desired service from said viewer;

said identifying comprises identifying said plurality of points of interest as points of interest providing said service and located within said desired distance.

13. The computerized method according to claim 1, further comprising updating at least one of:

said current view direction, said current location, a viewer'"'"'s viewing angle, a viewer'"'"'s distance from a surface onto which said plurality of location graphic symbolic indicators are projected, and a position of a vehicle and angle within a scene, to yield an updated 3D model and updated position of said plurality of location graphic symbolic indicators.

14. The computerized method according to claim 1, further comprising calculating importance and distance of each of said plurality of points;

wherein each one of said plurality of location graphic symbolic indicators is sized according to the importance of said respective point of interest.

15. The computerized method according to claim 1, wherein each of said plurality of location graphic symbolic indicators is an arrow having a direction repeatedly updated based on a current position of said vehicle and on a calculated route between said vehicle and one of said plurality of points of interest.

16. The computerized method of claim 1, further comprising estimating a 3D volume adoption the driver'"'"'s head and adapting said superimposing according to said 3D volume adoption.

17. The computerized method of claim 1, wherein said plurality of location graphic symbolic indicators are projected on a head up display (HUD) display.

18. The computerized method of claim 1, wherein at least some of said plurality of points are acquired from a remote web server based on said current location.

19. The computerized method of claim 1, wherein at least some of said plurality of points of interest are acquired from a collision avoidance system (CAS) installed in said vehicle.

20. The computerized method of claim 1, wherein at least some of said plurality of points of interest are acquired from a training module generating said plurality of points of interest to emulate a location of a plurality of other vehicles.

21. A non transitory computer readable medium comprising computer executable instructions adapted to perform the method of claim 1.

22. A system comprising:

a processor; and

a projector,

wherein the processor is configured to;

estimate a current location of a viewer in a vehicle;

estimate a current view direction of a viewer in said vehicle; and

identify a plurality of specified positions of a plurality of points of interest according to said estimated current view direction and said current location;

superimpose on a see through substance an image having a plurality of location graphic symbolic indicators, each one of said plurality of location graphic symbolic indicators is superimposed to appear over a section of a building viewed via said see through substance in one of said plurality of specified positions;

wherein each one of said plurality of location graphic symbolic indicators is sized according to a distance between said current location and a respective point of interest of said plurality of points of interest and a direction from said current location to said respective point of interest.