Methods and systems for augmented navigation

• US 8,265,866 B2
• Filed: 12/15/2010
• Issued: 09/11/2012
• Est. Priority Date: 12/15/2010
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Altamura, Francesco, Cortesi, Simone
• Primary Examiner(s): Camby, Richard M.
• Application Number:
  US12/968,918
• Publication Number:
  US 20120158287A1
• Time in Patent Office: 636 Days
• Field of Search: 701/400, 701/408, 701/409, 701/420, 701/423, 701/426, 701/432, 701/438, 701/457, 701/461, 34099526-99528
• US Class Current: 701/409
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3682 output of POI information on a road map G01C21/3614, G01C21/3685 take precedence
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

A navigation device is described that includes a processing device, a memory communicatively coupled to the processing device, the memory including a database of cartographic navigation map data, a first sensor operable to provide location data to the processing device, a display operatively coupled to the processing device, and an image acquisition device operatively coupled to the processing device. The processing device is programmed to, when in a first mode of operation, to cause the display to display a portion of the cartographic navigation map data, the displayed portion based on the location data received by the processing device. The processing device is further configured, when in a second mode of operation, to cause the display to display an image acquired by the image acquisition device, the image overlaid by one or more symbols indicative of one or more points of interest within the database of cartographic navigation map data, the one or more symbols correlated to the acquired image based on the location data received by the processing device.

Claims

1. A navigation device comprising:

a processing device;

a memory communicatively coupled to said processing device, said memory comprising a database of cartographic navigation map data;

a first sensor operable to provide location data to said processing device;

a display operatively coupled to said processing device;

a gyroscope; and

at least one image acquisition device operatively coupled to said processing device, said processing device programmed to, when in a first mode of operation, to cause said display to display a portion of the cartographic navigation map data, the displayed portion based on the location data received by said processing device, and said processing device configured, when in a second mode of operation, to cause said display to display an image acquired by said at least one image acquisition device, the image overlaid by one or more symbols indicative of one or more points of interest within the database of cartographic navigation map data, the one or more symbols correlated to the acquired image based on the location data received by said processing device, said at least one image acquisition device being operable to provide a view angle and a focal length of a lens associated with said at least one image acquisition device to said processing device, said gyroscope being operable to determine which direction the lens of said at least one image acquisition device is directed, the directional information calculated by said processing device using received gyroscope data, and said processing device being programmed to correlate the view angle, focal length, and directional information with the map data in said database to assign a coordinate to each symbol on said display by calculating its relative position to an edge of the acquired image.

2. The navigation device of claim 1 further comprising a second sensor, an orientation of said navigation device, as sensed by said second sensor, operable for switching said navigation device between the first mode of operation and the second mode of operation.

3. The navigation device of claim 2 wherein the first mode of operation is enabled by said second sensor sensing said navigation device is in a horizontal position, and the second mode of operation is enabled by said second sensor sensing said navigation device is in a vertical position.

4. The navigation device of claim 1 wherein at least one of said display and said at least one image acquisition device are remotely located from said navigation device.

5. The navigation device of claim 1 further comprising a switch, said switch operable for switching said processing device from a first mode of operation to a second mode of operation.

6. The navigation device of claim 1 further comprising a user input interface, said interface operable for selecting an overlaid point of interest displayed on said display, said processing device programmed to overlay additional data related to the selected point of interest on said display.

7. The navigation device of claim 1 wherein said at least one image acquisition device comprises at least one of a digital camera, a video camera, an infrared-camera, and a low-light night vision camera.

8. The navigation device of claim 1 further comprising a communications interface communicatively coupled to said processing device, said communication interface operable for receiving sensor information associated with another said navigation device deployed remotely from said navigation device.

15. The navigation device of claim 1 further comprising a communications interface communicatively coupled to said processing device, said communication interface operable for receiving data related to another said navigation device deployed remotely from said navigation device.

9. An augmented navigation method for a navigation device comprising:

determining if the navigation device is in a map display mode or in an acquired image display mode;

if the navigation device is determined to be in a map display mode,

determining a location of the navigation device, and

displaying a portion of a cartographic map associated with the determined location; and

if the navigation device is determined to be in an acquired image display mode,

determining a location of the navigation device,

providing a view angle and a focal length of a lens associated with an image acquisition device coupled to the navigation device,

determining, using a gyroscope, which direction the lens of the image acquisition device is directed,

correlating the view angle, focal length, and directional information with the cartographic map data,

assigning a coordinate to each symbol on a display by calculating its relative position to an edge of an acquired image,

displaying the acquired image, and

displaying overlays of points of interest on the acquired image, the points of interest stored in the cartographic map data, the overlays correlated to be in a correct position with respect to the acquired image.

10. The augmented navigation method of claim 9 wherein determining if the navigation device is in a map display mode or in an acquired image display mode comprises:

receiving data from an orientation sensor within the navigation device; and

placing the navigation device into the map display mode or the acquired image display mode based on the received data.

11. The augmented navigation method of claim 9 wherein determining if the navigation device is in a map display mode or in an acquired image display mode comprises:

receiving an external input within the navigation device; and

placing the navigation device into the map display mode or the acquired image display mode based on the received input.

12. The augmented navigation method of claim 9 wherein determining a location of the navigation device comprises receiving location data from a GPS within the navigation device.

13. The augmented navigation method of claim 9 further comprising:

using a user interface to select one of the overlays; and

overlaying additional data relating to the selected point of interest.

14. A navigation device comprising:

a processing device;

a gyroscope;

a memory communicatively coupled to said processing device, said memory comprising a database of cartographic navigation map data;

a GPS receiver operable to provide location data to said processing device;

a display operatively coupled to said processing device;

at least one image acquisition device operatively coupled to said processing device; and

a sensor operable for determining an orientation of said navigation device, wherein in a first determined orientation, said processing device is in a first mode of operation and is programmed to cause said display to display a portion of the cartographic navigation map data based on location data received by said processing device, and wherein in a second determined orientation, said processing device is in a second mode of operation and is programmed to cause said display to display an image acquired by said at least one image acquisition device, the image overlaid by one or more symbols related to the cartographic navigation map data, said at least one image acquisition device being operable to provide a view angle and a focal length of a lens associated with said at least one image acquisition device to said processing device, said gyroscope being operable to determine which direction the lens of said at least one image acquisition device is directed, the directional information calculated by said processing device using received gyroscope data, and said processing device being programmed to correlate the view angle, focal length, and directional information with the map data in said database to assign a coordinate to each symbol on said display by calculating its relative position to an edge of the acquired image.

16. The navigation device of claim 14 wherein said processing device is programmed to utilize data received from said at least one gyroscope to switch said navigation device between the display of cartographic navigation map data and an image acquired by said at least one image acquisition device.

17. The navigation device of claim 14 further comprising a user input interface, said interface operable for selecting an overlaid symbol displayed on said display, said processing device programmed to overlay additional data related to a point of interest represented by said symbol.

18. The navigation device of claim 14 wherein said at least one image acquisition device comprises at least one of a digital camera, a video camera, an infrared-camera, and a low-light night vision camera one or more of which may be remote from said navigation device.