Augmented Reality Control Systems

• US 20140111544A1
• Filed: 03/15/2013
• Published: 04/24/2014
• Est. Priority Date: 10/24/2012
• Status: Active Grant

• Current Assignee: Elbit Systems of America LLC (Elbit Systems Limited)
• Original Assignee: Exelis Incorporated (L3Harris Technologies, Inc.)
• Inventors: Hammond, John B.
• Granted Patent: US 9,129,429 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

An augmented reality control system in which the generation and utilization of simple geometric forms of real world objects at the enterprise level are layered onto or based on real world objects. Digital data is aligned to real world objects via surrogate of simplified geometric forms.

Claims

1. An augmented reality control system comprising:

a mobile device including;

an image capture device operable to capture an image of a real scene,

a location sensor operable to determine the global position of the mobile device, and

an orientation sensor operable to determine the orientation of the mobile device; and

an enterprise server including geometry data based on a digital coordinate system, the enterprise server being in wireless communication with the mobile device,

wherein the mobile device receives the geometry data from the enterprise server and applies the geometry data to the image.

2. The augmented reality control system of claim 1, wherein the orientation sensor is a sensor suite including a compass and an inertial measurement unit.

3. The augmented reality control system according to claim 2, wherein the location sensor is a global positioning system.

4. The augmented reality control system according to claim 3, wherein the mobile device is a head-up display.

5. The augmented reality control system of claim 4, wherein the mobile device is configured to filter the geometry data by performing hidden line removal perspective setting on the geometry data based on the pointing vector and location of the mobile device.

6. The augmented reality control system of claim 5, wherein the mobile device is further operable to perform geometry matching, matching the digital data coordinates with the image captured by the image capture device.

7. A method of augmenting a real scene captured by a mobile device, the method comprising:

(a) determining a location of the mobile device via a global positioning sensor;

(b) determining a pointing vector of the mobile device via an orientation sensor;

(c) wirelessly transmitting the pointing vector of the mobile device to an enterprise server operable to register geometry data based on digital coordinate system with an image captured by the mobile device;

(d) transmitting geometry data to the mobile device; and

(e) at the mobile device;

(e.1) filtering the geometry data based on the pointing vector of the mobile device and the device location, and

(e.2) grabbing a video frame of the real scene; and

(e.3) applying the filtered geometry data to the grabbed video frame.

8. The method according to claim 7 further comprising continuously comparing the digital coordinate system with the filtered geometry and device location information.

9. The method of claim 7, wherein the orientation sensor is a sensor suite including a compass and an inertial measurement unit.

10. The method of claim 7, wherein the mobile device is a head-up display.

13. The method of claim 7, wherein the orientation sensor is a sensor suite including a compass and an inertial measurement unit.

14. The method of claim 7, wherein the mobile device is a head-up display.

11. A method of augmenting a real scene captured by a mobile device, the method comprising:

(a) determining a location of the mobile device via a global positioning sensor;

(b) capturing a video frame of an image;

(c) determining a pointing vector of the mobile device via an orientation sensor;

(d) wirelessly transmitting the image, pointing vector, and location of the mobile device to an enterprise server operable to register geometry data based on digital coordinate system with an image captured by the mobile device;

(d) at the enterprise server, identifying and filtering geometry data based on the pointing vector and device location;

(e) overlaying the filtered geometry data on the video frame and performing geometry matching;

(f) applying the filtered geometry data to the video image; and

(g) transmitting geometry data to the mobile device.

12. The method according to claim 11 further comprising:

(h) sending digital information from the enterprise server to the mobile device; and

(i) displaying digital data on the mobile device such that it overlays the real scene.