Augmented vision for survey work and machine control

• US 6,094,625 A
• Filed: 07/03/1997
• Issued: 07/25/2000
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Ralston, Stuart E.
• Primary Examiner(s): SHAH, KAMINI S
• Application Number:
  US08/887,964
• Time in Patent Office: 1,118 Days
• Field of Search: 702/150, 364/188, 364/138, 239/161, 703/702, 367/436, 701/207, 434/29, 434/43, 434/49
• US Class Current: 702/150
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Methods and apparatus which enable use of augmented vision in survey procedures and related work. Virtual objects are presented to an operator on a see through display. Some of the objects correspond with real objects in the operators field of view at any instant and others may provide interactive functions for the operator. The operator eye positions are determined using measurements from a remote positioning system such as GPS, an operator head orientation sensing system, and knowledge of head geometry. Positions and attributes of real objects are stored in a database memory carried by the operator. The operators field of view is continuously determined and images of the real objects are generated from database information by a rendering system. The images are presented on the see through display as virtual objects. Numerous functions may be provided for the operator using virtual interactive objects, depending on the work at hand. Functions to assist navigation around a site, measurement of new survey points, virtual entry of control commands, site checking, and survey calculations such as intersections and offsets may be provided. One adaptation of the augmented vision system relates to machine control and enables an operator to guide a vehicle or the use of mechanical equipment.

Claims

1. A method for measuring a position of a survey point comprising:

determining a current field of view of an operator;

determining an initial position in the current field of view for a virtual point locator;

displaying an image containing a representation of the virtual point locator at the initial position according to the current field of view;

determining successive offset positions for the virtual point locator;

displaying successive images of the virtual point locator according to the current field of view and the offset positions; and

selectively storing positions of the virtual point locator as survey points.

2. The method as set forth in claim 1, wherein the steps of displaying display images of the virtual point locator on see-through display such that the virtual point locator is superimposed on a real image seen through the current view field of view.

3. The method as set forth in claim 1, wherein the step of determining an initial position for a virtual point locator comprises the steps of measuring a current position of a physical point locator and determining the initial position as an offset from the physical point locator.

4. The method as set forth in claim 3, wherein the physical point locator is selected from the group comprising a range pole and coupled positioning system antenna, and a head mounted positioning system antenna.

5. The method as set forth in claim 1, wherein the images of the virtual point locator comprise a visual representation of a range pole when in the field of view.

6. The method as set forth in claim 1, further comprising the step of receiving offset commands from the operator through a virtual controller device, said offset commands determining successive offset positions.

7. The method as set forth in claim 1, further comprising the step of regenerating the virtual point locator at the initial position in response to a reset command.

8. A method for controlling a survey operation comprising the steps of:

receiving a command from an operator to receive input;

determining a current field of view for the operator;

determining a display position in the current field of view for a virtual control object;

displaying an image comprising a representation of the virtual control object at the display position, said image including operator input options corresponding to the virtual control object; and

activating a survey apparatus to implement a selected input option in response to selection of the selected input option.

9. The method as set forth in claim 8, wherein the step of displaying displays images of the virtual control object on see-through display such that the virtual control object is superimposed on a real image seen through the current view field of view.

10. The method as set forth in claim 8, wherein the step of determining a position for a virtual control object comprises the steps of measuring a current position of a physical point locator and determining the position as an offset from the physical point locator.

11. The method as set forth in claim 10, wherein the physical point locator is selected from the group comprising a range pole and coupled positioning system antenna, and a head mounted positioning system antenna.

12. The method as set forth in claim 8, wherein the representation of the virtual control object is selected from the group comprising a representation of a survey control device and a hand held controller having a keypad and a display.

13. The method as set forth in claim 8, wherein further comprising the step of the operator selecting a selected input option using a pointing device.

14. The method as set forth in claim 13, further comprising the step of monitoring a glove worn by the operator and using a finger position of the operator to select the input option.

15. A method for viewing data comprising:

measuring a current position of an operator on a site using a position survey system;

measuring a current head orientation of the operator;

determining a current field of view of the operator through the display according to the current position and current head orientation measurements;

presenting an image containing a representation of at least one satellite in the remote positioning system which are within the field of view.

16. The method as set forth in claim 15, further comprising:

determining a mask elevation below which satellite signals are ignored when position measurements are taken; and

presenting an image containing a representation of the mask elevation.

17. A method for viewing information during a procedure for determining locations on a site comprising:

measuring a current position of an operator on a site;

measuring a current head orientation of the operator;

determining a current field of view of the operator through the display according to the current position and current head orientation measurements;

displaying an image containing at least one representation of a location of a point on a see through display oriented in the operator'"'"'s current field of view such that the at least one representation is superimposed at a position in the field of view, said displaying comprising;

displaying an image at the location of the point if the point is within the current field of view, and

displaying an image comprising an indicator as to where the point lies on the site fi the point is not within the current field of view; and

generating and presenting subsequent images of representation of a location of a point as the operator navigates around the site and the current field of view changes.

18. The method as set forth in claim 17, wherein the procedure comprises a survey to check a design, said method further comprising the step of referencing a predetermined layout of locations of a plurality of features in the design, wherein an image displayed at the location comprises a representation corresponding to the feature.

19. A system for measuring a position of a survey point comprising:

a display configured to display images relative to a current field of view of an operator;

a storage device configured to store survey points;

a controller coupled to the display and the storage device, said controller configured to determine a current field of view of an operator, determine an initial position in the current field of view for a virtual point locator, display an image on the display, said image containing a representation of the virtual point locator at the initial position according to the current field of view, determine successive offset positions for the virtual point locator, display successive images of the virtual point locator according to the current field of view and the offset positions, and selectively store in the storage device positions of the virtual point locator as survey points.

20. The system as set forth in claim 19, wherein the display comprises a heads up see through display configured such that the virtual point locator is viewed as superimposed on a real image seen through the current view field of view.

21. The system as set forth in claim 19 further comprising a physical point locator, said controller further configured to determine an initial position for the virtual point locator according to the current position of a physical point locator and the initial position as an offset from the position of the physical point locator.

22. The system as set forth in claim 21, wherein the physical point locator is selected from the group comprising a range pole and coupled positioning system antenna, and a head mounted positioning system antenna.

23. The system as set forth in claim 19, wherein the images of the virtual point locator comprise a visual representation of a range pole when in the field of view.

24. The system as set forth in claim 19, further comprising a virtual controller device configured to receive offset commands from the operator, said offset commands utilized to determine successive offset positions.

25. The system as set forth in claim 19, further comprising the step of regenerating the virtual point locator at the initial position in response to a reset command.

26. A system for controlling a survey operation comprising:

an input device configured to receive a command from an operator;

a current field of view of the operator;

a display configured to display an image of a representation of the virtual control object at the display position, said image including operator input options corresponding to the virtual control object; and

a survey apparatus coupled to the input device, said survey apparatus configured to implement a selected input option in response to selection of the selected input option through the input device.

27. The system as set forth in claim 26, wherein the displays comprises a heads up see through display such that images of the virtual control object are superimposed on a real image seen through the current view field of view.

28. The system as set forth in claim 26, further comprising a physical point locator, said the location of the virtual control object based upon a current position of a physical point locator.

29. The system as set forth in claim 28, wherein the physical point locator is selected from the group comprising a range pole and coupled positioning system antenna, and a head mounted positioning system antenna.

30. The system as set forth in claim 26, wherein the representation of the virtual control object is selected from the group comprising a representation of a survey control device and a hand held controller having a keypad and a display.

31. The system as set forth in claim 26, further comprising a pointing device configured to enable the operator to select a selected input option.

32. The system as set forth in claim 31, further wherein the pointing device is selected from the group comprising a glove and a mouse.

33. A system for viewing data comprising:

a position survey system configured to measure a current position of an operator on a site;

a measurement device configured to measure a current head orientation of the operator; and

a display configured to provide a current field of view of the operator through the display according to the current position and current head orientation measurements, said display further configured to display an image containing a representation of at least one satellite in the remote positioning system which is within the field of view.

34. The system as set forth in claim 33, further comprising a mask elevation measurement device which determines an elevation below which satellite signals are ignored when position measurements are taken, said display further configured to display an image containing a representation of the mask elevation.