Method and apparatus for position based query with augmented reality headgear

• US 10,740,502 B2
• Filed: 07/08/2019
• Issued: 08/11/2020
• Est. Priority Date: 02/22/2017
• Status: Active Grant

• Current Assignee: Middle Chart, LLC
• Original Assignee: Middle Chart, LLC
• Inventors: Duff, Jason E., Santarone, Michael S., Wodrich, Michael A., Pugh, Randall
• Primary Examiner(s): Laughlin, Nathan L
• Application Number:
  US16/504,919
• Publication Number:
  US 20200034501A1
• Time in Patent Office: 400 Days
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G01S 5/02 using radio waves using satellite radio beacon systems for determining position G01S19/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: Y02P 90/84 Greenhouse gas [GHG] management systems

Abstract

Methods and apparatus for presenting data to a user with an augmented reality headgear that has been oriented in a direction based upon unique automated generation of a vector. Systems for determining a direction of interest are based upon orientation of an apparatus. Data collection systems such as cameras, stereoscopic cameras, and scanner systems may be included in the augmented reality headgear. In combination with a geospatial position, the direction of interest may be referenced in the provision of content via a user interface displayed in a worn augmented reality headgear of the user at the site of interest. Updating of model data may be coordinated through the interface of an oriented augmented reality headgear with data collection systems.

Claims

1. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent in communication with a second transceiver supported by a hand of the agent at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and a first set of three reference point transceivers;

c. with a controller, determining X, Y and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of transceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector, wherein the direction of interest is the same as direction of the vector derived by a position of the hand of the agent supporting the second position determining device relative to the augmented reality headgear; and

h. generating a user interface viewable via the augmented reality headgear and comprising a human readable representation of the agent location and the direction of interest.

2. The method of claim 1 additionally comprising the steps of:

b. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver; and

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest.

3. The method of claim 2 wherein the direction of interest is at an angle to the direction of the vector that is generally perpendicular to the vector.

4. The method of claim 2 additionally comprising the step of calculating a distance from the headgear along a linear path based upon the direction of interest.

5. The method of claim 4 wherein the distance comprises a length of the vector.

6. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent and a second transceiver at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and a first set of three reference point transceivers;

c. with a contoller, determining X, Y, and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of transceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector;

h. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver; and

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest wherein the direction of interest is the same as direction of the vector derived by movement of a hand of a user while the hand is perceived by a stereoscopic camera system attached to the augmented reality headgear.

7. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent and a second transceiver at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and first set of three reference point transceivers;

c. with a controller, determining X, Y and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of transceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector;

h. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver; and

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest wherein the direction of interest is at an angle to the direction of the vector derived by movement of a hand of a user while the hand is perceived by a stereoscopic camera system attached to the augmented reality headgear.

8. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent and a second transceiver at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and first set of three reference point transceivers;

c. with a controller, determining X, Y and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of transceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector;

h. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver;

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest;

j. calculating a distance from the headgear along a linear path based upon the direction of interest, wherein the distance comprises a length of the vector;

k. selecting a virtual model of a structure based upon at least one of the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

l. accessing a first virtual position within the virtual model via projecting within the virtual model along a second vector comprising a direction congruent with the direction of interest; and

m. displaying, in the headgear, portions of the selected virtual model based upon the second vector.

9. The method of claim 8 wherein a length of the second vector is based upon a distance in the direction of interest to a surface of a first virtual item within the virtual model.

10. The method of claim 8 wherein the first virtual item comprises a machine.

11. The method of claim 8 wherein the first virtual item comprises an architectural feature.

12. The method of claim 8 additionally comprising the step of capturing image data with a stereoscopic camera supported by the augmented reality headgear and conveying the image data to the virtual model.

13. The method of claim 12 wherein the stereoscopic camera is positioned with a field of view comprising the direction of interest.

14. The method of claim 8 additionally comprising the step of capturing scan data with a three-dimensional scanner supported by the augmented reality headgear and conveying the scan data to the virtual model.

15. The method of claim 8 further comprising the step of modifying content displayed in the headgear based upon input from a glove comprising a haptic device, said glove wearable by the agent.

16. The method of claim 8 further comprising the step of modifying content displayed in the headgear based upon input from an accelerometer fixedly attached to the headgear.

17. The method of claim 8 further comprising the step of modifying the direction of interest based upon input from an accelerometer fixedly attached to the headgear.

18. The method of claim 8 further comprising the step of modifying the direction of interest based upon input from a user interface.

19. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent and a second transceiver at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and first set of three reference point transceivers;

c. with a controller, determining X, Y and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of transceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector;

h. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver;

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest;

j. calculating a distance from the headgear along a linear path based upon the direction of interest, wherein the distance comprises a length of the vector;

k. selecting a virtual model of a structure based upon at least one of the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

n. identifying a virtual position of the agent within the virtual model, the virtual position of the agent based upon at least one of the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

o. specifying a distance from the agent based upon a drop-down menu of items intersecting the second vector; and

p. portraying in the augmented reality headgear a portion of the virtual model from a perspective of the virtual position of the agent within the virtual model.

20. A method of transceiving geospatially oriented data, the method comprising the steps of:

a. supporting an augmented reality headgear on a head of an agent, said augmented reality headgear comprising a first transceiver at a first position relative to the head of the agent and a second transceiver at a second position relative to the head of the agent, while the augmented reality headgear is supported by the agent;

b. generating logical communications between the first transceiver and first set of three reference point transceivers;

c. with a controller, determining X, Y and Z coordinates of the first transceiver based upon respective logical communications between the first transceiver and each of tranceivers comprising the first set of three reference point transceivers;

d. generating respective logical communications between the second transceiver and a second set of three reference point transceivers;

e. with the controller, determining X, Y and Z coordinates of the second transceiver based upon respective logical communications between the second transceiver and each of the transceivers comprising the second set of three reference point transceivers;

f. generating a vector based upon the X, Y and Z coordinates of the first transceiver and X, Y and Z coordinates of the second transceiver;

g. generating a direction of interest based upon the vector;

h. generating a location of the agent based upon at least one of;

X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver;

i. generating a user interface comprising a human readable representation of the agent location and the direction of interest;

j. calculating a distance from the headgear along a linear path based upon the direction of interest, wherein the distance comprises a length of the vector;

q. selecting a virtual model of a structure based upon a location determined by a GPS device attached to the headgear;

r. accessing a first virtual position within the virtual model via projecting within the virtual model along a second vector comprising a direction congruent with the direction of interest; and

s. displaying, in the headgear, portions of the selected virtual model based upon the second vector.