Computer interface for navigating graphical user interface by touch

• US 7,379,053 B2
• Filed: 10/25/2002
• Issued: 05/27/2008
• Est. Priority Date: 10/27/2001
• Status: Expired due to Term

• Current Assignee: Vortant Technologies, LLC
• Original Assignee: Vortant Technologies, LLC
• Inventors: Schaefer, Philip
• Primary Examiner(s): Liang; Regina
• Application Number:
  US10/280,581
• Publication Number:
  US 20030090474A1
• Time in Patent Office: 2,041 Days
• Field of Search: 345173-179, 345156-157, 345/168, 345/169, 178/18.01, 178/18, 178/18.03, 341/22, 341/23
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A system for enabling a user to navigate a graphical user interface by touching pads on a touch surface. The arrangement of the pads on the touch surface is static, but the functions associated with each pad dynamically change in correspondence with functions in the graphical user interface. The functions are automatically and dynamically extracted from the software that creates the window display. Certain pads are regularly used for certain user options found in windows.

Claims

1. A system for interfacing with a computer having a software application running thereon, the software application having a plurality of user options, the system comprising:

an extraction module in communication with the software application that extracts information from the software application and associated with one or more of the plurality of user options displayed on a window of a computer interface associated with the computer, the extracted information including extracted window coordinates and extracted labels for each of the one or more of the plurality of user options;

a touch surface including an array of pads that can be touched by a user;

a mapping module including;

a position calculating module that calculates a relative position between the extracted window coordinates of a first one of the plurality of user options and the extracted window coordinates of at least a second one of the plurality of user options, and

a matching module that associates the first one of the plurality of user options with a first pad on the array of pads based on a relative position between the first pad and a second pad on the array of pads and based on the calculated relative position between the extracted windows coordinates for the first one and the second one of the plurality of user options; and

a touch module that, when the user touches the first pad in the array of pads, outputs to the user text associated with the extracted label of the first user option associated with the touched pad by the mapping module.

2. The system recited in claim 1, wherein the position calculating module includes a position categorization module that categorizes relative positions of the user options by symbolically describing vertical and horizontal relationships among rectangles defining locations and dimensions of the categorized user options.

3. The system recited in claim 2, wherein the symbolically described relationships include at least one of above, below, overlapping vertically, right-of, left-of, and overlapping horizontally.

4. The system recited in claim 1, wherein the matching module includes a search module that tests a plurality of permutations associating user options with pads, and for each tested permutation, the search module calculates a similarity of the calculated relative position between the user options and the calculated relative position between the pads associated with the user options, the matching module further including a selection module that selects one of the tested permutations as the association of a user option with a pad.

5. The system recited in claim 1, wherein the array of pads is contained in a housing that is distinct from a housing of the computer on which the software application is running, the system further comprising an electrical communication link between the array of pads and the computer.

6. The system recited in claim 5, wherein the electrical communication link is a wireless link.

7. The system recited in claim 5, wherein the electrical communication link is a cable.

8. The system recited in claim 1, wherein the matching module associates a subset of the user options to one or more of the pads, the matching module further including a scrolling function that associates a different subset of the user options to the one or more pads when activated by the user, whereby the software application may contain a greater number of user options than a number of pads in the array of pads, and whereby the user can access all of the user options without changing the user options being displayed.

9. The system recited in claim 8, wherein the scrolling function is activated by the user when the user touches a scrolling pad.

10. The system recited in claim 8, wherein the scrolling function is activated by the user when the user strokes a series of pads.

11. A method for interfacing a visually-impaired user with a software application operating on a computer, the software application having a plurality of user options, the method comprising:

extracting information from the software application, the extracted information associated with one or more of the plurality of user options displayed on a window of a user interface associated with the software application, the extracted information including extracted window coordinates and extracted labels for each of the one or more of the plurality of user options;

providing an array of touch pads by which the user may interface with the computer;

calculating a relative position between the extracted window coordinates of a first one of the plurality of user options and the extracted window coordinates of a second one of the plurality of user options;

mapping the first one of the plurality of user options with a first pad on the array of touch pads based on a relative position between the first pad and a second pad on the array of touch pads and based on the calculated relative position between the extracted windows coordinates for the first one and the second one of the plurality of user options; and

in response to the user touching the first pad, providing a response to the software application indicative of the user selecting the first one of the plurality of user options.