AUGMENTED REALITY VISUALIZATION SYSTEM

• US 20160379414A1
• Filed: 05/11/2016
• Published: 12/29/2016
• Est. Priority Date: 05/11/2015
• Status: Active Grant

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Brown, Christopher A., Juhl, Matthew
• Granted Patent: US 10,114,127 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: F41G 3/02 using an independent line of sight
• CPC Class Codes: F41G 3/06 with rangefinder rangefinders per se G01C
• CPC Class Codes: F41G 3/142 based on observation of a first shoot; using a simulated shoot training F41G3/26
• CPC Class Codes: F41G 3/165 using a TV-monitor
• CPC Class Codes: G01S 19/47 the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
• CPC Class Codes: G01S 19/53 Determining attitude
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G02B 27/0189 Sight systems
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

An augmented reality (AR) system comprising a head mounted display (HMD) configured to display one or more AR visualizations within an operator'"'"'s field of view (FOV), a control system including a processor and a storage system configured to store machine readable instructions, sensors configured to determine at least location and/or orientation of said sensors including a head mounted and device mounted sensor, and a communication system configured to communicate data between elements of the AR system. The software including various subroutines or machine readable instructions including an orientation/location instructions for determining orientation and/or position of the sensors, a visualizations generation instructions section configured to generate a visualization showing an aim point of a device coupled to said device mounted sensor, a path of travel of a projectile launched from said device, or an impact point of said projectile. Embodiments can include one or more photogrammetry processing sections.

Claims

1. An augmented reality (AR) system comprising:

a head mounted display (HMD) comprising a visual display system and a head mounting structure adapted to fix the HMD with respect to a human operator'"'"'s field of view (FOV), and a camera system oriented towards said FOV configured to capture a plurality of video images of entities or objects within the FOV;

a plurality of sensors comprising a head mounted position sensor and a first position sensor receiver positioned on a first device, said plurality of sensors comprising one or more internal navigation units (INU) or an internal measuring unit (IMU), said INU comprising a plurality of accelerometers, said IMU comprising an INU and a global position system (GPS) configured to determine a location of the GPS system, wherein said INU and IMU are configured to determine an orientation of at least a first device each said sensor is coupled with and output of said location as an orientation data;

a control system comprising a storage medium comprising a plurality of machine readable instructions operable to control elements of said AR system, a processor adapted to execute said plurality of machine readable instructions, said control system is configured to control said AR system, said sensors, and said HMD; and

a communication system configured to couple said HMD, said sensors, and said control system;

wherein said plurality of machine readable instructions comprises;

a first plurality of instructions configured to control said processor to operate said plurality of sensors to generate data used by said control system to determine three dimensional orientations of each of said sensors based determinations of location of the devices relative to at least the head mounted position system (HMPS) and said device mounted position sensor;

a second plurality of instructions configured to operate a video processing system to determine and create a wire frame model data of objects within the FOV based on the plurality of video images using a photogrammetry processing system

a third plurality of instructions configured to operate laser range finder and target designator system as well as an impact point determination system based on at least a first, second, and third data associated with at least one said first device, wherein said first data comprises an aim point of said first device, said second data comprises a computed impact point of a projectile fired from said first device, said third data comprises a path of travel of said projectile from a point of origin from said first device to said impact point, wherein said first data, second and third data each comprise one or more geo-reference system position coordinates;

a fourth plurality of instructions configured to control said processor to operate said HMD to generate and display on said HMD a plurality of AR visualizations comprising a first visualization within said FOV of said operator comprising at least a first and second visualization element based on a first, second, and third plurality of processing instruction outputs, said first visualization element comprises a visual designator showing said aim point on said wire frame model data, said impact point on said wire frame model data, or said path of travel with respect to at least a portion of said wire frame model data; and

a fifth plurality of instructions configured to provide a graphical user interface (GUI) control interfaces configured to create a control interface adapted to enable said operator to select elements of said first visualization to generate or control a plurality of AR visualization elements that said AR system generates on said HMD comprising said first visualization.