Electro-optic vision system which exploits position and attitude

• US 5,815,411 A
• Filed: 09/10/1993
• Issued: 09/29/1998
• Est. Priority Date: 09/10/1993
• Status: Expired due to Fees

• Current Assignee: Geovector Corporation
• Original Assignee: Geovector Corporation
• Inventors: Ellenby, John, Ellenby, Thomas William
• Primary Examiner(s): Voeltz, Emanuel T.
• Assistant Examiner(s): KEMPER, MELANIE A
• Application Number:
  US08/119,360
• Time in Patent Office: 1,845 Days
• Field of Search: 364/443, 364/449, 364/559, 364/578, 364/424.02, 395/118, 395/125, 395/127, 395/135, 395/133, 345/7, 345/8, 345/9, 348/115
• US Class Current: 702/150
• CPC Class Codes: G01C 17/34 Sun- or astro-compasses
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces

Abstract

The present invention is generally concerned with electronic vision devices and methods, and is specifically concerned with image augmentation in combination with navigation, position, and attitude devices. In the simplest form, devices of the invention can be envisioned to include six major components: A 1) camera to collect optical information about a real scene and present that information as an electronic signal to; a 2) computer processor; a 3) device to measure the position of the camera; and a 4) device to measure the attitude of the camera (direction of the optic axis), thus uniquely identifying the scene being viewed, and thus identifying a location in; a 5) data base where information associated with various scenes is stored, the computer processor combines the data from the camera and the data base and perfects a single image to be presented at; a 6) display whose image is continuously aligned to the real scene as it is viewed by the user.

The present invention is a vision system including devices and methods of augmented reality wherein an image of some real scene is altered by a computer processor to include information from a data base having stored information of that scene in a storage location that is identified by the real time position and attitude of the vision system. It is a primary function of the vision system of the invention, and a contrast to the prior art, to present augmented real images and data that is continuously aligned with the real scene as that scene is naturally viewed by the user of the vision system. An augmented image is one that represents a real scene but has deletions, additions and supplements.

Claims

1. An electro-optic apparatus for producing an augmented image from an image of a real scene and a computer generated image, the augmented image being aligned to the scene such that the directions up, down, left and right of the augmented image correspond directly with up, down, left and right of the real scene and the normal direction of the image plane points in a direction away from the user and towards the scene, said augmented image being comprised of information generated by an imaging means and information recalled from a data store, the information being particular to the position as determined by the apparatus and attitude of the apparatus, which comprises:

an imaging means;

a position determining means;

an attitude determining means;

a data store;

a computer; and

a display,

said imaging means having an optic axis which is a symmetry axis and defines the direction of viewing, a lens symmetric about that axis which defines an image field, and a charge coupled device in the image field;

said position determining means having a reference point corresponding to said imaging means;

said attitude determining means having a reference direction parallel to the direction of viewing of said imaging means;

said data store having memory locations wherein prerecorded data are stored, a plurality of orthogonal variables which identify those memory locations, and a pointer responsive to values from the position and attitude determining means which selects particular memory locations containing particular prerecorded data corresponding to the real scene;

said computer being in electronic communication with said imaging means, position and attitude determining means, data base, and display, having graphics and image processing capabilities; and

said display being an elecro-optic emissive display having a normal direction orthogonal to the display plane, the normal direction being colinear with the symmetry axis of the imaging means and being in communication with said computer.

2. The apparatus in claim 1 where the display is integrated into a binocular type viewing device having an optical path which corresponds to each of two eyes.

3. The apparatus of claim 1 further comprising an optics system of lenses to image the display at optical infinity to give the user the appearance and feeling of actually looking at the real scene as opposed to looking at a display device in the near field thereby allowing the user to relax the eye muscles by focusing at infinity.

4. The apparatus of claim 1 said computer being responsive to image features of the real scene as indicated by some predetermined condition.

5. An electro-optic apparatus for producing an augmented image from an image of a real scene and a computer generated image, the augmented image being aligned to the scene such that the directions up, down, left and right of the augmented image correspond directly with up, down, left and right of the real scene and the normal direction of the image plane points in a direction away from the user and towards the scene, said augmented image being comprised of information of generated by an imaging means and information recalled from a data store, the information being particular to the position as determined by the apparatus and attitude of the apparatus, which comprises:

an imaging means;

a position determining means;

an attitude determining means;

a data store;

a computer; and

a display,

said imaging means being sensitive to photon input and operable for converting light into an electronic signal processable by said computer;

said positioning determining means being operable for determining the location of the imaging means and presents a value which represents that location to the computer;

said attitude determining means being operable for determining the attitude of the imaging means as defined by the axis of the imaging means and presents a value which represents that attitude to the computer;

said data store being operable for storing prerecorded data corresponding to and representing objects known to be in the present field of view of the imaging means as defined by the present position and attitude thereof, recalling that data and transmitting it to the computer;

said computer being operable for receiving signals from imaging means and data store and combining those signals such that a composite image signal is formed and further operable for transmitting the composite image to said display;

said display being an elecro-optic emissive display operable for converting the composite image signal of the computer to a physical signal viewable by a user that is aligned to the viewing axis of said imaging means, the electro-optic vision apparatus allows the user to see his environment with computer generated augmented images aligned to the scene as the user would view the scene normally.