Speech supported navigation of a pointer in a graphical user interface

• US 5,920,841 A
• Filed: 06/25/1997
• Issued: 07/06/1999
• Est. Priority Date: 07/01/1996
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Schwab, Viktor, Schottmuller, Claus
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US08/882,667
• Time in Patent Office: 741 Days
• Field of Search: 704/275, 345/157
• US Class Current: 704/275
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

A method and an apparatus for speech controlled navigation of a pointer in a graphical user interface. Previous methods use speech commands like arrow keys of a keyboard and lack user friendly interfaces. The method and apparatus therefore provides a space of discrete position states (quantization) for the pointer which can be navigated only via those discrete positions by means of speech command input. The granularity of the discrete position states can be adapted to the respective application window and the position states can be represented by a system of coordinates where the speech command input is based on absolute or relative coordinates. Advantageously a copy image of the graphical user interface of operation can be provided in front of or beside the actual user interface and a proxy pointer device is displayed on the copy image. In one embodiment, only the copy image comprises the discrete position states, and the speech input commands are only transferred to the copy image. Navigation of the proxy pointer device within the copy is transferred and converted into commands within the actual user interface. By this method, an operation event effected by a manipulation of the proxy pointer effects a corresponding event at the user interface.

Claims

1. A method for speech-supported navigation of a cursor within a graphical user interface, said method comprising the steps of:

defining a coordinate system including a plurality of discrete position states within said graphical user interface;

scaling a granularity of said coordinate system in response to a first speech command; and

navigating said cursor among said plurality of discrete position states in response to a second speech command.

2. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 1, further comprising the step of manipulating said graphical user interface in response to said navigating step.

3. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 1, said scaling step comprising the step of modifying a minimum length of said coordinate system.

4. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 1, said method further comprising the step of displaying a coordinate grid including a plurality of intersection points corresponding to said plurality of discrete position states.

5. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 4, said navigating step comprising the step of navigating said cursor among said plurality of discrete position states in response to a second speech command including one of said plurality of intersection points.

6. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 1,

said establishing step comprising the step of establishing an absolute coordinate for each of said plurality of discrete position states;

said navigating step comprising the step of navigating said cursor among said plurality of discrete position states in response to a second speech command including one of said plurality of absolute coordinates.

7. A method for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 1, said method further comprising the steps of:

displaying a copy of said graphical user interface;

displaying a proxy cursor within said copy of said graphical user interface;

indicating a current position state of said cursor from among said plurality of discrete position states utilizing said proxy cursor;

said navigating step comprising the steps of;

navigating said proxy cursor in response to said second speech command; and

navigating said cursor among said plurality of discrete position states in response to said step of navigating said proxy cursor.

8. A data processing system for speech-supported navigation of a cursor within a graphical user interface comprising:

means for generating a copy of said graphical user interface;

means for generating a proxy cursor within said copy of said graphical user interface;

means for simultaneously displaying said copy of said graphical user interface and said proxy cursor;

means for navigating said proxy cursor within said copy of said graphical user interface in response to a speech command; and

means for navigating said cursor in response to a navigation of said proxy cursor.

9. A data processing system for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 8, said means for navigating said proxy cursor within said copy of said graphical user interface in response to a speech command comprising:

means for defining a coordinate system including a plurality of discrete position states within said copy of said graphical user interface;

means for scaling a granularity of said coordinate system in response to a first speech command; and

means for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command.

10. A data processing system for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 9, said means for scaling comprising means for modifying a minimum length of said coordinate system.

11. A data processing system for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 9, further comprising:

means for displaying a coordinate grid within said copy of said graphical user interface including a plurality of intersection points corresponding to said plurality of discrete position states;

said means for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command comprising means for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command including one of said plurality of intersection points.

12. A data processing system for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 9, further comprising means for indicating a current position state of said cursor from among said plurality of discrete position states utilizing said proxy cursor.

13. A data processing system for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 8, further comprising:

means for generating an operation event within said graphical user interface in response to a navigation of said proxy cursor; and

means for displaying an operation event within said copy of said graphical user interface in response to a generation of an operation event within said graphical user interface.

14. A computer program product to be utilized within a data processing system for speech-supported navigation of a cursor within a graphical user interface comprising:

media usable by said data processing system for storing and executing instruction means;

instruction means, embodied within said media, for generating a copy of said graphical user interface;

instruction means, embodied within said media, for generating a proxy cursor within said copy of said graphical user interface;

instruction means, embodied within said media, for simultaneously displaying said copy of said graphical user interface and said proxy cursor;

instruction means, embodied within said media, for navigating said proxy cursor within said copy of said graphical user interface in response to a speech command; and

instruction means, embodied within said media, for navigating said cursor in response to a navigation of said proxy cursor.

15. A computer program product for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 14, said instruction means for navigating said proxy cursor within said copy of said graphical user interface in response to a speech command comprising:

instruction means, embodied within said media, for defining a coordinate system including a plurality of discrete position states within said copy of said graphical user interface;

instruction means, embodied within said media, for scaling a granularity of said coordinate system in response to a first speech command; and

instruction means, embodied within said media, for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command.

16. A computer program product for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 15, said instruction means for scaling comprising instruction means, embodied within said media, for modifying a minimum length of said coordinate system.

17. A computer program product for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 15, further comprising:

instruction means, embodied within said media, for displaying a coordinate grid within said copy of said graphical user interface including a plurality of intersection points corresponding to said plurality of discrete position states;

said instruction means for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command comprising instruction means, embodied within said media, for navigating said proxy cursor among said plurality of discrete position states in response to a second speech command including one of said plurality of intersection points.

18. A computer program product for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 15, further comprising instruction means, embodied within said media, for indicating a current position state of said cursor from among said plurality of discrete position states utilizing said proxy cursor.

19. A computer program product for speech-supported navigation of a cursor within a graphical user interface as set forth in claim 14, further comprising:

instruction means, embodied within said media, for generating an operation event within said graphical user interface in response to a navigation of said proxy cursor; and

instruction means, embodied within said media, for displaying an operation event within said copy of said graphical user interface in response to a generation of an operation event within said graphical user interface.