Methods and apparatus for providing navigational information associated with locations of objects

• US 9,080,881 B2
• Filed: 06/12/2014
• Issued: 07/14/2015
• Est. Priority Date: 09/11/2009
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: France, Peter, Sharp, Kevin, Ralston, Stuart
• Primary Examiner(s): TISSOT, ADAM D
• Application Number:
  US14/302,890
• Publication Number:
  US 20140365116A1
• Time in Patent Office: 397 Days
• Field of Search: 701/213, 701/408, 701/538, 701/541, 701/487, 345/633, 345/629, 345/632
• US Class Current: 1/1
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence

Abstract

An apparatus for providing navigational information associated with locations of objects includes an imaging device configured to acquire image data, a visual display coupled to the imaging device and configured to display the image data, a position measuring device configured to determine position information associated with the imaging device, and an orientation device configured to determine orientation information associated with the imaging device. The apparatus may also include a rendering system coupled to the visual display, the position measuring device, and the orientation device. The rendering system may be configured to determine image coordinates associated with a location of an object and provide a navigational graphic on the visual display oriented relative to the image coordinates.

Claims

1. A handheld apparatus for providing navigational information associated with locations of objects, comprising:

an imaging device configured to acquire image data, the imaging device calibrated to enable transformations between three-dimensional coordinates of the objects in a surrounding environment and corresponding two-dimensional image coordinates on an image plane;

a visual display coupled to the imaging device and configured to display the image data;

a position measuring device configured to determine a position of the handheld apparatus;

one or more tilt sensors configured to determine a pitch and a roll of the handheld apparatus;

an orientation device configured to determine an orientation of the handheld apparatus; and

a rendering system coupled to the visual display, the position measuring device, the one or more tilt sensors, and the orientation device, the rendering system configured to;

obtain three-dimensional coordinates associated with a location of an object in the surrounding environment;

determine two-dimensional image coordinates on the image plane that are associated with the three-dimensional coordinates of the object; and

provide a navigational graphic overlaid on the image data displayed on the visual display, the navigational graphic positioned on the visual display relative to the two-dimensional image coordinates to identify the location of the object relative to the image data.

2. The handheld apparatus of claim 1 wherein the two-dimensional image coordinates on the image plane that are associated with the three-dimensional coordinates of the object are determined using calibration of the imaging device, the position of the handheld apparatus, the pitch and the roll of the handheld apparatus, the orientation of the handheld apparatus, and the three-dimensional coordinates associated with the location of the object in the surrounding environment.

3. The handheld apparatus of claim 1 wherein the rendering system is further configured to determine that the location of the object is captured by the image data displayed on the visual display, and configured to overlay the navigational graphic on the image data associated with the location.

4. The handheld apparatus of claim 1 wherein the rendering system is further configured to determine that the location of the object is not captured by the image data displayed on the visual display, and configured to overlay the navigational graphic on the image data to indicate the location relative to the image data.

5. The handheld apparatus of claim 1 wherein the rendering system is further configured to determine that the location of the object is not captured by the image data displayed on the visual display, and configured to provide the navigational graphic in the shape of an arrow overlaid on the image data to indicate the location relative to the image data.

6. The handheld apparatus of claim 1 wherein the visual display is configured to display the image data in real-time, and the rendering system is configured to determine the two-dimensional coordinates in real-time and to provide the navigational graphic in real-time.

7. The handheld apparatus of claim 1 wherein the rendering system is configured to provide the navigational graphic in the shape of an arrow arranged relative to the image data associated with the location of the object.

8. The handheld apparatus of claim 1 wherein the visual display is wirelessly coupled to the imaging device.

9. The handheld apparatus of claim 1 wherein the position measuring device comprises a GNSS receiver.

10. The handheld apparatus of claim 1 further comprising an input device for data input to the handheld apparatus.

11. A handheld apparatus for providing navigational information associated with locations of objects, comprising:

an imaging device configured to acquire image data, the imaging device calibrated to enable transformation between three-dimensional coordinates of the objects in a surrounding environment and corresponding two-dimensional image coordinates on an image plane;

a visual display coupled to the imaging device and configured to display the image data;

a position measuring device disposed in a fixed position relative to the imaging device and configured to determine a position of the imaging device;

a rotation sensor configured to determine rotation of the imaging device about horizontal axes;

an orientation device configured to determine an orientation of the imaging device about a vertical axis; and

one or more processors coupled to the visual display, the position measuring device, the rotation sensor, and the orientation device, the one or more processor configured to;

obtain three-dimensional coordinates associated with a location of an object in the surrounding environment;

determine two-dimensional image coordinates on the image plane that are associated with the three-dimensional coordinates of the object using calibration of the imaging device, the position of the imaging device, the rotation of the imaging device, the orientation of the imaging device, and the three-dimensional coordinates associated with the location of the object in the surrounding environment; and

provide a navigational graphic overlaid on the image data displayed on the visual display, the navigational graphic positioned on the visual display relative to the two-dimensional image coordinates to identify the location of the object relative to the image data.

12. The handheld apparatus of claim 11 wherein the one or more processors are further configured to determine that the location of the object is captured by the image data displayed on the visual display, and configured to overlay the navigational graphic on the image data associated with the location.

13. The handheld apparatus of claim 11 wherein the one or more processors are further configured to determine that the location of the object is not captured by the image data displayed on the visual display, and configured to overlay the navigational graphic on the image data to indicate the location relative to the image data.

14. A method of providing navigational information associated with locations of objects, the method comprising:

acquiring image data using an imaging device, the imaging device calibrated to enable transformations between three-dimensional coordinates of objects in a surrounding environment and corresponding two-dimensional image coordinates on an image plane;

displaying the image data using a visual display;

determining a position of the imaging device;

determining rotation of the imaging device about horizontal axes;

determining an orientation of the imaging device;

obtaining three-dimensional coordinates associated with a location of an object in the surrounding environment;

determining two-dimensional image coordinates on the image plane that are associated with the three-dimensional coordinates of the object; and

providing a navigational graphic overlaid on the image data displayed on the visual display, the navigational graphic positioned on the visual display relative to the two-dimensional image coordinates to identify the location of the object relative to the image data.

15. The method of claim 14 wherein the two-dimensional image coordinates on the image plane that are associated with the three-dimensional coordinates of the object are determined using calibration of the imaging device, the position of the imaging device, the rotation of the imaging device, the orientation of the imaging device, and the three-dimensional coordinates associated with the location of the object in the surrounding environment.

16. The method of claim 14 wherein the navigational graphic is superimposed on the image data.

17. The method of claim 14 wherein the navigational graphic is in the shape of an arrow.

18. The method of claim 14 further comprising determining that the location of the object is captured by the image data displayed on the visual display, and overlaying the navigational graphic on the image data associated with the location.

19. The method of claim 14 further comprising determining that the location of the object is not captured by the image data displayed on the visual display, and overlaying the navigational graphic on the image data to indicate the location relative to the image data.

20. The method of claim 14 wherein the image data is displayed in real-time, and the two-dimensional coordinates are determined in real-time and the navigational graphic is provided in real-time.