Graphical user interfaces for computer vision systems

• US 6,414,696 B1
• Filed: 09/04/1998
• Issued: 07/02/2002
• Est. Priority Date: 06/12/1996
• Status: Expired due to Fees

• Current Assignee: Geo Vector Corporation
• Original Assignee: Geo Vector Corporation
• Inventors: Page, Joseph, Ellenby, John, Ellenby, Thomas, Ellenby, Peter
• Primary Examiner(s): SAX, STEVEN PAUL
• Application Number:
  US09/146,820
• Time in Patent Office: 1,397 Days
• Field of Search: 345/326-358, 345/762, 345/763, 345/825, 345/826, 345/650, 345/661, 345/667, 345/656, 345/726, 345/725, 345/719, 345/720, 345/721, 345/722, 345/723, 345/724, 340/211-214, 707/10
• US Class Current: 715/762
• CPC Class Codes: G01C 17/34 Sun- or astro-compasses
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: Y10S 715/972 Inserted real-time video in operator interface

Abstract

Computer vision systems provide a user a view of a scene whereby an image of the scene may have been augmented with information generated by a computer. Computer vision systems of the present invention include graphical user interfaces which have been discovered to operably interact with geometric constructs of a user environment, objects within a scene, perspective of the scene, image features of a signal which represents the scene, among others. These graphical user interfaces of the invention do not behave as those known because operation of these interfaces depends on properties and features particular to computer vision systems which have position and attitude determining means.

Claims

1. An apparatus for generating graphical user interfaces, the apparatus comprising:

an attitude determining means;

a position determining means;

a computer;

a graphics generator; and

a display,

said computer being electronically coupled to said display, graphics generator, and attitude determining means,

said graphics generator being responsive to said attitude and position determining means.

2. An apparatus of claim 1, further comprising a switch coupled to said computer, said switch being operable for allowing a user to select a position and attitude measurement at an instant in time.

3. A graphical user interface for a computer system, the computer system having a computer;

position and attitude determining means; and

a display, said graphical user interface being comprised of;

a field region; and

a periphery,

said field region being an area fractional portion of the display field enclosed by said periphery operable for displaying image and graphical information while a scene is being addressed by said computer system.

4. A graphical user interface of claim 3, additionally comprising a response mechanism, said response mechanism being in communication with said field region whereby said response mechanism operates to change the information content of the graphical user interface.

5. A graphical user interface of claim 4, said response mechanism electronically connected to said position and attitude means whereby position and attitude information is transmitted to said response mechanism and the content of the graphical user interface is responsive to position and attitude determinations.

6. A graphical user interface of claim 4, said graphical user interface additionally comprising a sub-field region, said sub field region being responsive to a cursor.

7. A graphical user interface of claim 6, said cursor being indicia corresponding to a system pointing direction.

8. A graphical user interface of claim 4, comprising a sub-field region responsive to the position of the computer vision system.

9. A graphic user interface of claim 8, comprising pointing indicia with a reference point.

10. A graphic user interface of claim 9, comprising a sub-field region responsive to the location of the reference point with respect to an image being displayed.

11. A graphical user interface of claim 3, said periphery corresponding in shape to a three dimensional object in the scene as it appears in perspective from the point-of-view of the computer system.

12. A graphical user interface of claim 3, further having indicia, for example a color highlighted region responsive to the location of the reference point with respect to the image of the scene being addressed.

13. A graphical user interface of claim 3, said field region contains a plurality of sub-fields each contained entirely within the periphery of said graphical user interface.

14. A graphical user interface of claim 13, said plurality of sub-fields comprises at least one sub-field which corresponds in shape to a three dimensional object in the scene being addressed as it appears from the point-of-view of the computer system.

15. A graphical user interface of claim 3, comprising indicia in the shape of a polygon having at least three sides, each side of the polygon corresponding to a side of the periphery of the field region or the boundary of the display field, the periphery of the field region corresponding to a physical feature of the scene being addressed as it appears from the point-of-view of the computer system.

16. A method of displaying a graphical user interface in a computer system comprising the steps:

addressing a scene with a computer system;

determining the position and attitude of the computer system;

generating a graphical user interface in accordance with said position and attitude determination; and

displaying the graphical user interface as an image.

17. A method of claim 16, the graphical user interface being dynamically movable with respect to the display field.

18. A method of claim 17, comprising the steps:

engaging the graphical user interface with a cursor;

activating a switch;

dragging the graphical user interface across the display field while holding the switch active; and

releasing the switch.

19. A method of claim 18, said releasing the switch causes the graphical user interface to be locked to a point in the display field whereby the graphical user interface is associated with a position corresponding to a point in a displayed image.

20. A method of claim 18, said releasing the switch causes the graphical user interface to be locked to a direction in the display field whereby the graphical user interface is associated with a direction of the compass.

21. A method of claim 16, the ‘

generating a graphical user interface’

step being further defined as providing a graphical user interface having associated therewith a plane and normal direction, associating the normal direction with a particular direction in space and displaying the graphical user interface in perspective with respect to the point-of-view of the computer.

22. A method of claim 17, where ‘

dynamically movable’

refers to movable about a reference point and is realized executing the following steps;

locking a graphical user interface point to a point in an image;

engaging the graphical user interface with a cursor whereby the cursor is caused to become co-located with the interior region of the graphical user interface;

activating a switch which causes the cursor to become locked to the field region of the graphical user interface;

advancing the cursor and consequently the field region about a curve; and

deactivating the switch causing the cursor to be released from the field region.

23. A method of claim 17, where ‘

dynamically movable’

refers to movable about a display field and is realized executing the following steps;

engaging the graphical user interface with a cursor to cause the cursor to become co-located with the interior area of the graphical user interface;

activating a switch to cause the cursor to become locked to the field region of the graphical user interface;

advancing the cursor and consequently the field region in any direction in the plane of the display field but without limits of a preselected area; and

deactivating the switch causing the cursor to become released from the field region.

24. A method of claim 17, where ‘

dynamically movable’

refers to movable about a display field and is realized executing the following steps;

tracking an object moving in a scene;

maintaining the position of the graphical user interface with respect to the moving object.

25. A method of claim 17, where ‘

dynamically movable’

refers to movable with respect to the attitude of the computer system whereby the graphical user interface is displayed on a direction defined by the half-angle between the attitude of the computer system and a reference direction.

26. A method of claim 17, where ‘

dynamically movable’

refers to movable within a plane in the scene being addressed which is movable within a plane in space such that panning of a reference direction causes the field of the graphical user interface to advance in the plane such that it has an angular appearance as the pan angle approaches 90 degrees.

27. A method of claim 16, said periphery is expanded in size such that it increases in size while the image behind remains the same size.

28. A method of claim 16, said periphery remains constant providing for an apparent size.

29. A method of claim 16, said periphery being defined by a click-and-drag process.