Augmented reality implement control

• US 8,918,246 B2
• Filed: 12/27/2012
• Issued: 12/23/2014
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Friend, Paul
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US13/728,536
• Publication Number:
  US 20140188333A1
• Time in Patent Office: 726 Days
• Field of Search: 701/50, 701/36, 701/408, 701/432, 701/436, 701/34.4, 345 7- 9, 345632-634, 345/641, 359/13
• US Class Current: 701/34.4
• CPC Class Codes: E02F 9/261 Surveying the work-site to be treated
• CPC Class Codes: E02F 9/264 Sensors and their calibration for indicating the position of the work tool
• CPC Class Codes: E02F 9/267 Diagnosing or detecting failure of vehicles
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

The disclosed strategy and method assists in controlling a machine equipped with a work implement for performing work at a worksite. The work implement can be positioned around the worksite such that the implement is obstructed from view through an operator display device through which the worksite is visible. A controller or the like determines the position of the work implement and generates an augmented overlay. The augmented overlay is displayed on the operator display device superimposed over the worksite indicating the position of the work implement.

Claims

1. A machine comprising:

a work implement having a working surface;

an operator display device in which at least a portion of the work implement is visible; and

a controller configured to generate an augmentation overlay indicating the working surface of the work implement and communicating the augmentation overlay for display on the operator display device.

2. The machine of claim 1, further comprising at least one feedback sensor for determining a position of the work implement, the at least one feedback sensor communicating with the controller.

3. The machine of claim 1, wherein the controller stores implement dimensional data regarding geometric proportions of the work implement and uses the implement dimensional data to generate the augmentation overlay.

4. The machine of claim 1, wherein the augmentation overlay is superimposed over a worksite visible in the operator display device.

5. The machine of claim 1, wherein the operator display device is selected from a group consisting of a head mounted display, a heads-up display and an off-board display.

6. The machine of claim 5, wherein the head mounted display and the heads-up display include a transparent display through which a worksite is visible.

21. The machine of claim 1, wherein the working surface of the work implement is obstructed from view in the operator display device.

7. A method of operating a work implement in a worksite, the method comprising:

positioning a work implement in a position in which a working surface of the work implement is obstructed from view through an operator display device;

generating an augmentation overlay indicating the working surface; and

displaying on the operator display device the augmentation overlay superimposed on the worksite.

8. The method of claim 7, further comprising:

determining the position of the work implement with at least one feedback sensor; and

processing the position determined to generate the augmentation overlay.

9. The method of claim 8, further comprising:

storing geometric information regarding geometric proportions of the work implement; and

processing the geometric information to generate the augmentation overlay.

10. The method of claim 9, wherein the augmentation overlay includes an implement overlay superimposed over the work implement.

11. The method of claim 10, wherein the implement overlay is selected from the group consisting of a wireframe image, shading or a plurality of dots.

12. The method of claim 7, further comprising:

repositioning the work implement in the worksite; and

regenerating the augmentation overlay to reflect repositioning of the work implement.

13. The method of claim 7, wherein the operator display device is selected from a group consisting of a head mounted display, a heads-up display and an off-board display.

14. The method of claim 7, further comprising including worksite information in the augmented overlay.

15. The method of claim 14, wherein the worksite information is selected from the group consisting of a desired terrain of the worksite, a worksite material characteristic, and a target point disposed in the worksite.

16. The method of claim 7, further comprising measuring a slippage value of a propulsion device of a machine associated with the work implement;

and inferring a quantity of material displaced by the work implement from the slippage value measured.

17. A controller-implemented method for assisting control of a work implement, the method comprising:

storing implement dimensional data regarding geometric extensions of a work implement;

receiving a plurality of data inputs including implement position data;

generating an implement overlay from the implement dimensional data and the implement position data;

displaying the implement overlay on an operator display device so as to be superimposed over the work implement.

18. The method of claim 17, wherein the plurality of data inputs are received from at least one feedback sensors operatively associated with the work implement.

19. The method of claim 17, wherein the implement overlay is selected from the group consisting of a wireframe image, shading or a plurality of dots.

20. The method of claim 17, wherein the operator display device is selected from a group consisting of a head mounted display, a heads-up display and an off-board display.