Method and Apparatus for Construction Machine Visualization

• US 20160004305A1
• Filed: 07/01/2015
• Published: 01/07/2016
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Topcon Positioning Systems Incorporated (Topcon Corporation)
• Original Assignee: Infomobility S.r.l., Tierra S.p.A (Topcon Corporation), Topcon Positioning Systems Incorporated (Topcon Corporation)
• Inventors: Jackson, Richard, Di Federico, Ivan, Pagliani, Daniele, Finardi, Nicola
• Granted Patent: US 10,503,249 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06Q 10/063114 Status monitoring or status determination for a person or group
• CPC Class Codes: G06Q 50/08 Construction
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 9/31 Projection devices for colour picture display , e.g. using electronic spatial light modulators [ESLM]

Abstract

A system for displaying information to an operator of a machine comprises a head tracking system and a projection system. One system for displaying information uses a projection system and a see-through display to present two-dimensional images to an operator. One system uses a projection system and a see-through display to present three-dimensional images to an operator. One system uses a pair of smart glasses to display information to a user based a direction a user is looking.

Claims

1. A method comprising:

determining a direction of a user'"'"'s gaze;

determining information to be displayed to the user based on the direction of the user'"'"'s gaze, a current state of a construction site, and a current state of a machine; and

displaying the information to the user.

2. The method of claim 1, wherein the direction of the user'"'"'s gaze is determined based on a head position of the user.

3. The method of claim 1, wherein the direction of the user'"'"'s gaze is determined based on a position of the user'"'"'s eyes and a direction the user'"'"'s eyes are looking.

4. The method of claim 1, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

determining an operation to be performed based on the difference, wherein the information comprises the operation to be performed.

5. The method of claim 1, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the current state of the construction site and the desired site plan,

wherein information comprises the percent of completion of the construction site.

6. The method of claim 1, further comprising:

determining that a user'"'"'s gaze has changed;

determining new information to be displayed to the user based on the determining that the user'"'"'s gaze has changed;

displaying the new information to the user.

7. The method of claim 6, further comprising:

determining that the current state of the construction site has changed,

wherein the determining new information to be displayed to the user is further based on a new state of the site.

8. The method of claim 1, wherein the displaying the information to the user comprises:

transmitting the information to be displayed to the user to a projection system.

9. The method of claim 1, wherein the displaying the information to the user comprises:

transmitting the information to be displayed to the user to a pair of smart glasses.

10. An apparatus comprising:

a processor; and

a memory to store computer program instructions, the computer program instructions when executed on the processor, cause the processor to perform operations comprising;

determining a direction of a user'"'"'s gaze;

determining information to be displayed to the user based on the direction of the user'"'"'s gaze, a current state of a construction site, and a current state of a machine; and

displaying the information to the user.

11. The apparatus of claim 10, wherein the direction of the user'"'"'s gaze is determined based on a head position of the user.

12. The apparatus of claim 10, wherein the direction of the user'"'"'s gaze is determined based on a position of the user'"'"'s eyes and a direction the user'"'"'s eyes are looking.

13. The apparatus of claim 10, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

determining an operation to be performed based on the difference, wherein the information comprises the operation to be performed.

14. The apparatus of claim 10, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the current state of the construction site and the desired site plan,

wherein information comprises the percent of completion of the construction site.

15. The apparatus of claim 10, the operations further comprising:

determining that a user'"'"'s gaze has changed;

determining new information to be displayed to the user based on the determining that the user'"'"'s gaze has changed;

displaying the new information to the user.

16. A computer readable medium storing computer program instructions, which, when executed on a processor, cause the processor to perform operations comprising:

determining a direction of a user'"'"'s gaze;

determining information to be displayed to the user based on the direction of the user'"'"'s gaze, a current state of a construction site, and a current state of a machine; and

displaying the information to the user.

17. The computer readable medium of claim 16, wherein the direction of the user'"'"'s gaze is determined based on a head position of the user.

18. The computer readable medium of claim 16, wherein the direction of the user'"'"'s gaze is determined based on a position of the user'"'"'s eyes and a direction the user'"'"'s eyes are looking.

19. The computer readable medium of claim 16, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

determining an operation to be performed based on the difference,

wherein the information comprises the operation to be performed.

20. The computer readable medium of claim 16, the determining information to be displayed to a user comprising:

determining a difference between the current state of the construction site and a desired site plan;

generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the current state of the construction site and the desired site plan, wherein information comprises the percent of completion of the construction site.