Electro-optic vision systems

• US 6,064,398 A
• Filed: 08/02/1996
• Issued: 05/16/2000
• Est. Priority Date: 09/10/1993
• Status: Expired due to Term

• Current Assignee: Geovector Corporation
• Original Assignee: Geovector Corporation
• Inventors: Ellenby, John, Ellenby, Thomas William, Ellenby, Peter Malcolm, McGuire, Kenyon E. B.
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US08/691,784
• Time in Patent Office: 1,383 Days
• Field of Search: 395/119, 395/135, 395/128, 345/435, 345/419, 345/428
• US Class Current: 345/633
• CPC Class Codes: G01C 17/34 Sun- or astro-compasses
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

An image processing system for producing an augmented image of a real world scene uses an image capturing device that captures a digital image of the real world scene and uses a database of real world scene information. The system includes the storage resources that locate the selected real world scene information in the database using information from at least one of a position determining device for determining a real world position, an attitude determining device for determining a real world attitude, and the image capturing device. The system also includes a video interface chipset for receiving data from the image capturing device and transforming it to a format used by the data processor. The system processes the digital image of the real world scene and the selected real world scene information to form processed data that represents an augmented image of the real world scene.

Claims

1. An image processing system for producing an augmented image of a real world scene using an image capturing device that captures a digital image of the real world scene and using a database of real world scene information, the image processing system comprising:

a data processor operatively connected to a 3D graphic processor;

first communication resources that deliver the digital image of the real world scene from the image capturing device to the data processor;

storage resources that retrieve from the database selected real world scene information that relates to the real world scene wherein the storage resources locate the selected real world scene information in the database using information from at least one of a position determining device for determining a real world position, an attitude determining device for determining a real world attitude, and the image capturing device;

second communication resources that deliver the selected real world scene information from the database to the data processor, wherein the second communication resources includes a video interface chipset for receiving data from the image capturing device and transforming said receiver data to a format used by the data processor, and wherein the data processor processes the digital image of the real world scene and the selected real world scene information to form processed data that represents an augmented image of the real world scene; and

a display system that receives the processed data and displays the augmented image.

2. The system of claim 1 wherein the storage resources comprise:

a storage system containing the database;

the position determining device; and

processing resources,

wherein the position determining device determines an approximate three-dimensional real world position of the image processing system, and the processing resources use the real world position to search the database and retrieve from the database the selected real world scene information related to the real world position.

3. The system of claim 2 wherein the position, determining device comprises a satellite navigation system.

4. The system of claim 3 wherein the satellite navigation system comprises a Global Navigational Satellite System.

5. The system of claim 3 wherein the satellite navigation system comprises a Global Positioning System.

6. The system of claim 3 wherein the satellite navigation system comprises a differential Global Positioning System.

7. The system of claim 2 wherein the database is Geographic Information System database.

8. The system of claim 1 wherein the storage resources comprise:

a storage system containing the database;

the attitude determining device; and

processing resources,

wherein the attitude determining device determines an approximate real world attitude of the image processing system, and the processing resources-use the real world attitude to search the database and retrieve from the database the selected real world scene information related to the attitude information.

9. The system of claim 8 wherein the attitude determining device comprises a tri-axial magnetometer.

10. The system of claim 1 wherein the real world scene information includes images and the storage resources further comprise:

processing resources that determine an approximate shape of an object captured by the image capturing device, and compare the approximate shape to the images in the database to find a close match, and include the close match in the selected real world scene information.

11. The system of claim 1 wherein the data processor and the 3D graphic processor are contained in a single integrated circuit having a central processing unit capable of executing a set of multimedia instructions.

12. The system of claim 1 further including one or more accelerometers for generating signals that are used by at least one of the data processor and the display system in image stabilization.

13. The system of claim 1 fturther including a wireless communication device that provides information to the data processor.

14. The system of claim 1 further including a range finder that provides information to the data processor.

15. The system of claim 1 further comprising at least one user interface device that provides information to the data processor.

16. The system of claim 1 wherein the data processor comprises a general purpose central processing unit running in parallel with the 3D graphic processor.

17. The system of claim 16 wherein the image processor enhances comprises means for enhancing the digital image of the real world scene.

18. The system of claim 16 wherein the image processor is adapted to perform a fast Fourier Transform.

19. The system of claim 1 wherein the display system comprises:

a display graphic processor for generating the augmented image from the processed data and for displaying the augmented image on a display.

20. The system of claim 1 wherein

the position determining device determines the real world position from the the real world position of the image processing system;

and

the attitude determining device determines the real world attitude from the real world attitude of the image image processing system.

21. The system of claim 1 wherein

the augmented image is formed and displayed substantially in real time.