Handheld virtual overlay system

• US 7,557,736 B1
• Filed: 08/31/2005
• Issued: 07/07/2009
• Est. Priority Date: 08/31/2005
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Daily, Michael J., Isdale, Jerry
• Primary Examiner(s): Goins; Davetta W
• Assistant Examiner(s): LAU, HOI CHING
• Application Number:
  US11/217,214
• Time in Patent Office: 1,406 Days
• Field of Search: 3409951-996, 340/990, 342/357.07, 342/357.13, 705/10, 701/201, 701/202, 701/207, 701/211, 701/213, 701/208, 202/158, 382/276
• US Class Current: 340/995.1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence

Abstract

A handheld virtual overlay system (HVOS) device is presented that includes a global positioning system (GPS) receiver for detecting a global position of the HVOS device; a compass/tilt sensor for detecting a direction and tilt of the HVOS device; a communication module for transmitting the global position, the direction, and the tilt to a data store; and an input/output display configured to receive and display a rendered image. The data store is configured to receive the global position, the direction, and the tilt, and to generate geo-coded data for transmission to a server, where the server is configured to receive the geo-coded data and to transform and render the geo-coded data into a rendered image. A user can point the HVOS device at a scene, with the HVOS device producing a rendered image representing the scene.

Claims

1. A handheld virtual overlay system (HVOS) device, comprising:

a global positioning system (GPS) receiver for determining a global position of the HVOS device;

a compass for detecting a direction of the HVOS device;

a communication module for transmitting the global position and the direction to a data store, the data store configured to receive the global position and the direction and to generate geo-coded data for transmission to a server, the server configured to receive the geo-coded data and to transform and render the geo-coded data into an egocentric rendered image representing the global position and orientation of the HVOS device; and

a display configured to receive and display the egocentric rendered image, whereby as a user points the HVOS device at a scene, the HVOS device produces and displays an egocentric rendered image representing the scene.

2. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising a tilt sensor for detecting a tilt of the HVOS device and configured such that the data store can receive the global position, the direction, and the tilt, and generate geo-coded data for transmission to a server where the server is configured to receive the geo-coded data and to transform and render the geo-coded data into the egocentric rendered image representing the global position, direction, and tilt of the HVOS device.

3. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising a data store communicatively connected with the communication module, the data store comprising geographic information system (GIS) data, geo-referenced images, and geo-referenced elevation information.

4. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising a server communicatively connected with the data store, the server comprising a geo-coding transformation engine and a three-dimensional (3D) renderer for rendering a rendered image.

5. A handheld virtual overlay system (HVOS) device as set forth in claim 1, wherein the communication module is a wireless communication module.

6. A handheld virtual overlay system (HVOS) device as set forth in claim 1, wherein the display includes an output mode for viewing the rendered image, and an input mode for interacting with the rendered image.

7. A handheld virtual overlay system (HVOS) device as set forth in claim 1, wherein the display is configured to display imagery while the handheld device is pointed at the imagery, and further includes a compass icon communicatively connected with the compass, the compass icon illustrating a directional relationship of the handheld device with the rendered image.

8. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising an output audio description of overlay data in connection with the compass.

9. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising a control module with at least three adjusters for adjusting the rendered image, a first adjuster being an overlay content adjuster, a second adjuster being a viewpoint adjuster, and a third adjuster being a realism adjuster.

10. A handheld virtual overlay system (HVOS) device as set forth in claim 1, further comprising a selection module communicatively connected with the display, the selection module configured to enable the system to receive and store personal preferences for each user.

11. A handheld virtual overlay system (HVOS) device as set forth in claim 10, wherein the selection module is further configured to cluster requests from multiple users.

12. A handheld virtual overlay system (HVOS) device as set forth in claim 10, wherein the selection module includes at least one pre-constructed profile, enabling a user to select a pre-constructed profile and thereby request default display parameters corresponding to the pre-constructed profile.

37. A handheld virtual overlay system (HVOS) device as set forth in claim 1, wherein the server is further configured to render an egocentric rendered image with an annotated horizon line.

13. A method for rendering an image representing a global position, comprising acts of:

determining a global position of a handheld virtual overlay system (HVOS) device;

detecting a direction of the HVOS device using a compass;

transmitting the global position and the direction to a data store using a communication module, the data store configured to receive the global position and the direction and to generate geo-coded data for transmission to a server, the server configured to receive the geo-coded data and to transform and render the geo-coded data into an egocentric rendered image representing the global position and orientation of the HVOS device; and

displaying the rendered image on a display, whereby as a user points the HVOS device at a scene, the HVOS device produces and displays an egocentric rendered image representing the scene.

14. A method for rendering an image representing a global position as set forth in claim 13, further comprising an act of detecting a tilt of the HVOS device using a tilt sensor, where the HVOS device is configured such that the data store can receive the global position, the direction, and the tilt, and generate geo-coded data for transmission to a server where the server is configured to receive the geo-coded data and to transform and render the geo-coded data into the egocentric rendered image representing the global position, direction, and tilt of the HVOS device.

15. A method for rendering an image representing a global position as set forth in claim 13, further comprising an act of communicatively connecting a data store with the communication module, the data store comprising geographic information system (GIS) data, geo-referenced images, and geo-referenced elevation information.

16. A method for rendering an image representing a global position as set forth in claim 13, further comprising an act of communicatively connecting a server with the data store, the server comprising a geo-coding transformation engine and a three-dimensional (3D) renderer for rendering a rendered image.

17. A method for rendering an image representing a global position as set forth in claim 13, wherein in the act of transmitting the global position and the direction to a data store using a communication module, the communication module is a wireless communication module.

18. A method for rendering an image representing a global position as set forth in claim 13, wherein in the act of configuring the display, the display includes an output mode for viewing the rendered image, and an input mode for interacting with the rendered image.

19. A method for rendering an image representing a global position as set forth in claim 13, wherein in the act of displaying the rendered image on a display, the display is configured to display imagery while the handheld device is pointed at the imagery, and further includes a compass icon communicatively connected with the compass, the compass icon illustrating a directional relationship of the handheld device with the rendered image.

20. A method for rendering an image representing a global position as set forth in claim 13, further comprising an act of outputting an audio description of overlay data.

21. A method for rendering an image representing a global position as set forth in claim 13, further comprising an act of adjusting the rendered image using a control module having three adjusters, a first adjuster being an overlay content adjuster, a second adjuster being a viewpoint adjuster, and a third adjuster being a realism adjuster.

22. A method for rendering an image representing a global position as set forth in claim 13, further comprising acts of receiving and storing personal preferences for each user using a selection module connected with the display.

23. A method for rendering an image representing a global position as set forth in claim 22, further comprising an act of clustering requests from multiple users through a solution module communicatively connected with the display.

24. A method for rendering an image representing a global position as set forth in claim 22, further comprising an act of using a selection module communicatively connected with the display to select a pre-constructed profile and thereby request default display parameters corresponding to the pre-constructed profile.

38. A method for rendering an image representing a global position as set forth in claim 13, wherein the server is further configured to render an egocentric rendered image with an annotated horizon line.

25. A computer program product for operating a virtual overlay system (HVOS) device, comprising instruction means encoded on a computer-readable medium, for causing operations of:

receiving a global position of the HVOS device from a global positioning system (GPS);

receiving a direction of the HVOS device from a compass;

using the global position and direction to generate geo-coded data;

transforming and rendering the geo-coded data into an egocentric rendered image representing the global position and orientation of the HVOS device; and

displaying the rendered image, whereby as a user points the HVOS device at a scene, the HVOS device produces and displays an egocentric rendered image representing the scene.

26. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for detecting a tilt of the HVOS device and configured such that the data store can receive the global position, the direction, and the tilt, and generate geo-coded data for transmission to a server where the server is configured to receive the geo-coded data and to transform and render the geo-coded data into the egocentric rendered image representing the global position, direction, and tilt of the HVOS device.

27. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for communicatively connecting a data store with the communication module, the data store comprising geographic information system (GIS) data, geo-referenced images, and geo-referenced elevation information.

28. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for communicatively connecting a server with the data store, the server comprising a geo-coding transformation engine and a three-dimensional (3D) renderer for rendering a rendered image.

29. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for configuring the display to include an output mode for viewing the rendered image, and an input mode for interacting with the rendered image.

30. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for configured the display to display imagery while the handheld device is pointed at the imagery, and further including a compass icon communicatively connected with the compass, the compass icon illustrating a directional relationship of the handheld device with the rendered image.

31. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for outputting audio descriptions of overlay data.

32. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for adjusting the rendered image using a control module having at least three adjusters, a first adjuster being an overlay content adjuster, a second adjuster being a viewpoint adjuster, and a third adjuster being a realism adjuster.

33. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for enabling the HVOS system to receive and store personal preferences for each user using a selection module communicatively connected with the display.

34. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 33, further comprising instruction means encoded on a computer-readable medium, for clustering requests from multiple users.

35. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 33, further comprising instruction means encoded on a computer-readable medium, for enabling a user to select a pre-constructed profile and thereby request default display parameters corresponding to the pre-constructed profile.

39. A computer program product for operating a handheld virtual overlay system (HVOS) device as set forth in claim 25, further comprising instruction means encoded on a computer-readable medium, for rendering an egocentric rendered image with an annotated horizon line.

36. A method for forming a handheld virtual overlay system (HVOS) device, comprising acts of:

configuring a global positioning system (GPS) receiver to determine a global position of the HVOS device;

configuring a compass for detecting a direction of the HVOS device;

configuring a communication module to transmit the global position and the direction to a data store, the data store configured to receive the global position and the direction and to generate geo-coded data for transmission to a server, the server configured to receive the geo-coded data and to transform and render the geo-coded data into an egocentric rendered image representing the global position and orientation of the HVOS device; and

configuring a display to receive and display the rendered image, whereby as a user points the HVOS device at a scene, and the HVOS device produces and displays an egocentric rendered image representing the scene.

40. A method for forming a handheld virtual overlay system (HVOS) device as set forth in claim 36, wherein the server is further configured to render an egocentric rendered image with an annotated horizon line.