Method and system for providing touch-sensitive screens for the visually impaired

• US 6,489,951 B1
• Filed: 01/07/1998
• Issued: 12/03/2002
• Est. Priority Date: 06/07/1995
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wong, Peter Kam-Ho, Cokus, Shawn J., Reifman, Jeffrey B., Lowney, Gregory C.
• Primary Examiner(s): Mengistu, Amare
• Application Number:
  US09/003,643
• Time in Patent Office: 1,791 Days
• Field of Search: 345/173, 345/123, 345/339, 345/340, 235/375, 235/379
• US Class Current: 345/173
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G09B 21/001 Teaching or communicating with blind persons G09B21/02 - G09B21/06 take precedence
• CPC Class Codes: H04M 1/2476 for a visually impaired user illuminating or arrangements for improving visibility of characters on dials H04M1/22

Abstract

A method and system for enabling a visually impaired user to use a touch-sensitive device. The present invention provides an interface for the visually impaired (IVI) system which enables a visually impaired user to use a touch-sensitive screen. The IVI system provides an explore mode in which a visually impaired user may determine which objects are on the touch-sensitive screen by dragging a finger over the touch-sensitive screen. In particular, the IVI system announces objects as a user touches them. For example, the IVI system announces text as it is touched and announces controls, such as buttons, as they are touched. In addition, the IVI system enables a user to execute controls. A user may execute a control, such as a button, by dragging a finger onto the control, and then, without dragging the finger off of the control, lifting the finger off of the touch-sensitive screen. The IVI system further enables a user to scroll through lists on a touch-sensitive screen. In particular, the IVI system enables a user to use a scroll thumb to scroll through a list. When a user drags a finger over the scroll thumb, and then, without dragging the finger off of the scroll thumb, lifts the finger off of the touch-sensitive screen, the IVI system enters scroll mode. In the scroll mode, a user may drag a finger anywhere on the touch-sensitive screen to scroll the list. While in scroll mode if a user again lifts their finger off of the touch-sensitive screen, the IVI system returns to explore mode.

Claims

1. A method in a computer system adapted for use by the visually impaired for announcing changes to objects displayed on a touch-sensitive screen, the method comprising the steps of:

displaying a plurality of objects on the touch-sensitive screen;

receiving a request to display a new object;

displaying the new object on the touch-sensitive screen;

retrieving information associated with the new object; and

automatically providing audio feedback associated with a description of the retrieved information.

2. The method of claim 1 wherein the step of retrieving information associated with the displaying of the new object is retrieved from a predefined location on the touch-sensitive screen.

3. The method of claim 1 wherein the new object is a new window.

4. A method in a computer system adapted for use by the visually impaired for announcing a description of objects displayed on a touch-sensitive screen, the method comprising the steps of:

receiving a request to display a new object on the touch-sensitive screen;

determining that a user has activated a summary mode for receiving the description of objects displayed on the touch-sensitive screen upon the new object being displayed on the touch-sensitive screen;

displaying the new object on the touch-sensitive screen; and

automatically providing audio feedback of the description of the objects displayed on the touch-sensitive screen.

5. The method of claim 4 wherein the new object is a new window.

6. A method in a computer system for selecting a list item of a list box, the computer system having a touch-sensitive screen, the method comprising the steps of:

displaying the list box containing list items, the list box and list items having associated coordinate positions on the touch-sensitive screen;

receiving a current coordinate position indicating where a user is touching the touch-sensitive screen or indicating that the user is not touching the touch-sensitive screen;

when the current coordinate position is associated with the list box and a last coordinate position received is not associated with the list box, providing audio feedback indicating that the user has entered the list box;

when the current coordinate position is associated with a list item and the last coordinate position was not associated with that list item, providing audio feedback indicating the list item; and

when the current coordinate position indicates the user is not touching the touch-sensitive screen and the last coordinate position was associated with the list item, selecting the list item.

7. The method of claim 6, when the current coordinate position indicates the user is not touching the touch-sensitive screen and the last coordinate position was associated with the list item, further including the following steps:

determining whether the list item is selected;

when the list item is selected,

deselecting the list item; and

providing audio feedback that the list item has been deselected; and

when the list item is not selected,

selecting the list item; and

providing audio feedback that the list item has been selected.

8. The method of claim 6, wherein the list box has a scroll bar region having a scroll box sub-region, including the steps of:

when the current coordinate position is associated with the scroll bar region and the last coordinate position is not associated with the scroll bar region, providing audio feedback indicating the user has entered the scroll bar region;

when the current coordinate position is associated with the scroll box sub-region and the last coordinate position is not associated with the scroll box sub-region, providing audio feedback indicating the user has entered the scroll box sub-region;

when the current coordinate position indicates the user is not touching the touch-sensitive screen and the last coordinate position was associated with the scroll box sub-region, entering a scroll mode; and

when in scroll mode, when the current coordinate position is different from the last coordinate position, scrolling the list items in the list box.

9. A computer system for selecting a control region of a touch-sensitive panel, the touch-sensitive panel having a plurality of control regions, the system comprising:

means for receiving an indication that a user is touching the touch-sensitive panel, the indication including a current coordinate position;

means for determining whether the current coordinate position is within a control region;

means for, when the current coordinate position is within the control region, providing audio feedback indicating that the current coordinate position is within the control region;

means for receiving an indication that the user is no longer touching the touch-sensitive panel while the current coordinate position is within the control region;

means for selecting the control region; and

means for, after selecting the control region, providing audio feedback indicating that the control region is selected.

10. A method in a computer system adapted for use by the visually impaired for announcing a description of a window displayed on a touch-sensitive panel, the method comprising:

receiving an indication that a summary mode has been activated;

receiving a request to display a change to the displayed window;

displaying the change to the displayed window; and

automatically providing audio feedback of a description of displayed changed window.

11. The method of claim 10 wherein the displayed change to the displayed window results in the display of a new window.

12. The method of claim 10 wherein the audio feedback describes the location and type of each object displayed in the displayed changed window.

13. The method of claim 10 wherein the request to display the change to the displayed window is a request to display a new window.

14. The method of claim 10, further comprising indicating activation of summary mode by inputting a key sequence.

15. The method of claim 10, further comprising indicating activation of summary mode by executing a control displayed on the touch-sensitive panel.

16. A computer-readable memory medium containing instructions for controlling a computer processor in a computer system adapted for use by the visually impaired to announce a description of a window displayed on a touch-sensitive panel each time the window changes by performing:

receiving an indication that a summary mode has been activated;

receiving an indication that the displayed window has been changed; and

after receiving the indication that the displayed window has been changed, automatically providing audio feedback that describes the displayed window.

17. The computer-readable memory medium of claim 16 wherein the audio feedback describes the type and location of objects in the displayed window.

18. The computer-readable memory medium of claim 16 wherein the received indication that the displayed window has been changed is an indication that a new window is being displayed.

19. The computer-readable memory medium of claim 17, further comprising indicating activation of summary mode by inputting a key sequence.

20. The computer-readable memory medium of claim 17, further comprising indicating activation of summary mode by executing a control displayed on the touch-sensitive panel.

21. A method in a computer system for selecting a control region of a touch-sensitive panel the touch-sensitive panel having a plurality of control regions, the method comprising:

determining whether a user is contacting the touch-sensitive panel over a control region; and

when the user has contacted the touch-sensitive panel over the control region,

providing audio feedback indicating that the user has entered the control region;

determining whether the user has terminated contact with the touch-sensitive panel by releasing contact from the touch-sensitive panel over the control region;

when the user has terminated contact with the touch-sensitive panel in such a way, selecting the control region, thereby causing the selection of the control region upon termination of contact instead of initiation of contact; and

providing audio feedback indicating that the control region was selected.

22. The method of claim 21 wherein the touch-sensitive panel is a touch-sensitive screen and further comprising displaying an indication of each of the control regions on the touch-sensitive screen.

23. The method of claim 21 wherein each of the control regions has an associated function, and further comprising:

executing the function associated with the selected one of the control regions; and

providing audio feedback indicating the execution of the function associated with the selected one of the control regions.

24. The method of claim 23 wherein the executing of the function associated with the selected one of the control regions is performed by contacting an activate control.

25. The method of claim 23 wherein the executing of the function associated with the selected one of the control regions is performed in response to lifting a finger from contact with the panel over the selected one of the control regions.

26. The method of claim 21 wherein the user contacts the touch-sensitive panel by touching a finger on the panel and releases contact from the touch-sensitive panel by lifting the finger from the panel.

27. The method of claim 21 wherein a portion of the control regions can be activated to execute a function and wherein an activatable control region is separated from another activatable control region by a control region that cannot be activated, and wherein the step of selecting the control region further executes the function associated with the control region when an activatable control region is selected.

28. The method of claim 27 wherein the control region that cannot be activated is static text.

29. The method of claim 27 wherein the control region that cannot be activated is a figure.