Augmented Reality Display System

• US 20150199106A1
• Filed: 01/14/2014
• Published: 07/16/2015
• Est. Priority Date: 01/14/2014
• Status: Abandoned Application

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Johnson, Lonny E.
• Application Number:
  US14/154,327
• Publication Number:
  US 20150199106A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A display system for use at a work site includes a display device having a display screen and a pose sensor system. A controller determines the position and orientation of the display device relative to the work site based upon the pose sensor system, generates machine control signals to control movement of a machine without an operator at the machine controlling the movement the machine, generates an augmented reality image based upon the machine control signals and the position and orientation of the display device, and renders the augmented reality image on the display screen.

Claims

1. A display system for use at a work site, comprising:

a display device including a display screen;

a pose sensor system associated with the display device for generating display device pose signals indicative of a position and an orientation of the display device relative to the work site; and

a controller configured to;

determine the position and orientation of the display device relative to the work site;

generate machine control signals to control movement of a machine without an operator at the machine controlling the movement the machine;

generate an augmented reality image based upon the machine control signals and the position and orientation of the display device; and

render the augmented reality image on the display screen.

2. The display system of claim 1, wherein the machine is being operated without an operator on the machine.

3. The display system of claim 1, wherein the controller generates the machine control signals remotely from the machine and transmits the machine control signals to the machine.

4. The display system of claim 1, wherein the controller is further configured to generate the machine control signals autonomously to autonomously control movement of the machine.

5. The display system of claim 1, wherein the controller is further configured to generate the machine control signals to control movement of the machine based upon signals provide by an operator located remotely from the machine.

6. The display system of claim 1, wherein the display device is a head mountable display device configured to be mounted on a user'"'"'s head.

7. The display system of claim 6, wherein the head mountable display device includes a transparent display screen on which the augmented reality image is displayed.

8. The display system of claim 1, wherein the display device is a portable computing device.

9. The display system of claim 8, wherein the controller is further configured to display visual images including images of the work site.

10. The display system of claim 1, wherein the display device is heads-up display device mounted on a second machine remote from the machine.

11. The display system of claim 1, wherein the controller is further configured to generate the augmented reality image based upon proximity of the display device to the machine.

12. The display system of claim 1, wherein the controller further includes a virtual work site map representing the work site, the virtual work site map including work site information, and generate the augmented reality image further based upon the work site information.

15. The method of claim 1, further including generating the machine control signals remotely from the machine and transmitting the machine control signals to the machine.

13. A controller-implemented method of operating a display system at a work site, comprising:

determining a position and an orientation of a display device relative to the work site based upon display device pose signals generated by a pose sensor system associated with the display device;

generating machine control signals to control movement of a machine without an operator at the machine controlling the movement the machine;

generating an augmented reality image based upon the machine control signals and the position and orientation of the display device; and

rendering the augmented reality image on a display screen of the display device.

14. The method of claim 13, further including operating the machine without an operator on the machine.

16. The method of claim 13, further including generating the machine control signals autonomously to autonomously control movement of the machine.

17. The method of claim 13, further including an operator located remotely from the machine generating the machine control signals to remotely control movement of the machine.

18. The method of claim 13, further including displaying the augmented reality image on a transparent display.

19. The method of claim 13, further including generating the augmented reality image based upon proximity of the display device to the machine.

20. A system for use at a work site, comprising:

a machine having a propulsion system, the machine being movable without an operator at the machine controlling movement of the machine;

a display device including a display screen;

a pose sensor system associated with the display device for generating display device pose signals indicative of a position and an orientation of the display device relative to the work site; and

a controller configured to;

determine the position and orientation of the display device relative to the work site;

generate machine control signals to control the movement of the machine;

generate an augmented reality image based upon the machine control signals and the position and orientation of the display device; and

render the augmented reality image on the display screen.