System and method for non-visually presenting multi-part information pages using a combination of sonifications and tactile feedback

• US 6,834,373 B2
• Filed: 04/24/2001
• Issued: 12/21/2004
• Est. Priority Date: 04/24/2001
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dieberger, Andreas
• Primary Examiner(s): Shah, Sanjiv
• Application Number:
  US09/840,227
• Publication Number:
  US 20020156807A1
• Time in Patent Office: 1,337 Days
• Field of Search: 715/501.1, 345/163, 345/156, 345/157, 345/173, 382/114, 434/114
• US Class Current: 715/247
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information

Abstract

A visually impaired user is provided a non-visual, abstract representation of a multi-part information page. The non-visual, abstract representation is provided by auditory and tactile feedback. A touch pad provides tactile feedback for the boundaries between content areas of the page when the user'"'"'s finger is located at coordinates on the touch pad which map to the boundaries between the content areas of the page. Auditory feedback provides meta-information on the content within a content area when the user'"'"'s finger is located at coordinates on the touch pad which map to the content area.

Claims

1. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, said method comprising:

maintaining a non-visual, abstract representation of said multi-part information page, said non-visual, abstract representation comprising non-visual display coordinates, said non-visual display coordinates comprising;

boundary coordinates defining boundaries between said two or more spatially located areas, said boundary coordinates associated with tactile feedback;

content coordinates defining said two or more spatially located areas, said content coordinates associated with auditory feedback, said auditory feedback representative of content meta-information;

receiving input position coordinates;

mapping said position input coordinates to said non-visual display coordinates;

generating a tactile output to cause a tactile output device to generate said tactile feedback for position input coordinates mapped to said boundary coordinates, and

generating an auditory output to cause an auditory output device to generate said auditory feedback for position input coordinates mapped to said content coordinates.

2. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 1, wherein said content meta-information indicates any of:

area updated;

area contains specific items designated of interest;

kind of content within area;

area is scrollable;

area is visible in its entirety;

number of hyperlinks in area;

area contains content in a visually impaired user inaccessible form;

or area contains content in a visually impaired user accessible form.

3. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 1, said method further comprising:

receiving data representative of said multi-part information page;

determining said two or more spatially located areas from said data;

determining said content meta-information for said separate content from said data, and

generating said non-visual, abstract representation from said area determination and said content meta-information determination.

4. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 3, said method further comprising:

caching said data representative of said multi-part information page.

5. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 4, said method further comprising:

receiving a request for said content in one of said areas;

retrieving said requested content from said cached data, and

transmitting said content to a linear screen reader.

6. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 5, wherein said multi-part information page is a web page containing frames.

7. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 4, said method further comprising:

receiving a request for said content in one of said areas;

retrieving said requested content from said cached data, and

generating a non-visual representation of said content displayable by said tactile output device and said auditory output device.

8. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 7, wherein said multi-part information page is a web page containing frames.

9. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 7, said method further comprising:

displaying said non-visual representation of said content via said tactile output device and said auditory output device.

10. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 1, wherein said multi-part information page is a web page containing frames.

11. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 1, wherein said multi-part information page is visually displayed.

12. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 11, wherein said non-visual, abstract representation is synchronized with said visually displayed multi-part information page.

13. A method of non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 1, wherein said tactile output device comprises any one of a touch pad, a bracelet, a ring, a necklace or a laser pointer.

14. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, said system comprising:

an intermediary, said intermediary generating a non-visual, abstract representation of said multi-part information page, said non-visual, abstract representation comprising non-visual display coordinates, said non-visual display coordinates comprising;

boundary coordinates defining boundaries between said two or more spatially located areas, said boundary coordinates associated with tactile feedback;

content coordinates defining said two or more spatially located areas, said content coordinates associated with auditory feedback, said auditory feedback representative of content meta-information;

a non-visual input/output interface;

said interface receiving said non-visual, abstract representation from said intermediary;

said interface receiving input position coordinates;

said interface mapping said position input coordinates to said non-visual display coordinates;

said interface generating a tactile output to cause a tactile output device to generate said tactile feedback for position input coordinates mapped to said boundary coordinates, and

said interface generating an auditory output to cause an auditory output device to generate said auditory feedback for position input coordinates mapped to said content coordinates.

15. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 14, wherein said content meta-information indicates any of:

area updated;

area contains specific items designated of interest;

kind of content within area;

area is scrollable;

area is visible in its entirety;

number of hyperlinks in area;

area contains content in a visually impaired user inaccessible form;

or area contains content in a visually impaired user accessible form.

16. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 14, wherein:

said intermediary receives a request for said content in one of said areas;

said intermediary retrieves said requested content from said cached data, and

said intermediary transmits said content to a linear screen reader.

17. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 14, wherein:

said intermediary receives a request for said content in one of said areas;

said intermediary retrieves said requested content from said cached data;

said intermediary generates a non-visual representation of said content displayable by said tactile output device and said auditory output device, and

said intermediary transmitting said non-visual representation of said content to said interface.

18. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 14, wherein said multi-part information page is a web page containing frames.

19. A system for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, as per claim 14, wherein said tactile output device comprises any one of a touch pad, a bracelet, a ring, a necklace or a laser pointer.

20. A system for facilitating collaboration between visually impaired and sighted users including a computer-based device, said device providing representations to said visually impaired users of a multi-part information page'"'"'s visual appearance by way of auditory and tactile feedback references indicating content layout and meta-information of content in a visually displayed multi-part information page, said system comprising:

a visual display, said display including a graphical representation, said graphical representation comprising one or more boundaries separating two or more content areas having distinct viewable content;

an electronic input device, said input device generating input position coordinates;

computer-based processor, said processor receiving said input position coordinates;

computer-based memory, said memory storing at least boundary coordinates corresponding to said visually displayed boundaries and content coordinates corresponding to said visually displayed content areas;

an auditory output device;

a tactile output device;

said computer-based processor comparing said stored coordinates with said input position coordinates;

said computer-based processor providing auditory feedback via said auditory output device representative of meta-information of viewable content in a content area for input position coordinates substantially equal to said content coordinates, and

said computer-based processor providing tactile feedback via said tactile output device for input position coordinates substantially equal to said boundary coordinates.

21. A system for facilitating collaboration between visually impaired and sighted users including a computer-based device, said device providing representations to said visually impaired users of a multi-part information page'"'"'s visual appearance by way of auditory and tactile feedback references indicating content layout and meta-information of content in a visually displayed multi-part information page, as per claim 20, wherein said electronic input device and said tactile output device is combined as an electronic touch pad, said touch pad generating said input position coordinates from user finger movements over various areas of a touch surface of said touch pad, said touch pad providing said tactile feedback in the form of vibrations.

22. A system for facilitating collaboration between visually impaired and sighted users including a computer-based device, said device providing representations to said visually impaired users of a multi-part information page'"'"'s visual appearance by way of auditory and tactile feedback references indicating content layout and meta-information of content in a visually displayed multi-part information page, as per claim 20, wherein said tactile output device comprises any one of a bracelet, a necklace or a laser pointer.

23. A system for facilitating collaboration between visually impaired and sighted users including a computer-based device, said device providing representations to said visually impaired users of a multi-part information page'"'"'s visual appearance by way of auditory and tactile feedback references indicating content layout and meta-information of content in a visually displayed multi-part information page, as per claim 20, wherein said multi-part information page is a web page containing frames.

24. An article of manufacture comprising a computer usable storage medium having computer readable data and instructions embodied therein for non-visually displaying a multi-part information page containing two or more spatially located areas of separate content, said computer readable data and instructions comprising:

computer readable data and instructions for maintaining a non-visual, abstract representation of said multi-part information page, said non-visual, abstract representation comprising non-visual display coordinates, said non-visual display coordinates comprising;

boundary coordinates defining boundaries between said two or more spatially located areas, said boundary coordinates associated with tactile feedback;

content coordinates defining said two or more spatially located areas, said content coordinates associated with auditory feedback, said auditory feedback representative of content meta-information;

computer readable data and instructions for receiving input position coordinates;

computer readable data and instructions for mapping said position input coordinates to said non-visual display coordinates;

computer readable data and instructions for generating a tactile output to cause a tactile output device to generate said tactile feedback for position input coordinates mapped to said boundary coordinates, and

computer readable data and instructions for generating an auditory output to cause an auditory output device to generate said auditory feedback for position input coordinates mapped to said content coordinates.

25. A method of non-visually representing graphical images by mapping included content sections and associated boundaries to non-visual feedback parameters, said image comprising a multiplicity of content sections and associated boundaries, said method comprising:

receiving a request from a requester to represent a specific instance of one of said graphical images;

generating a non-visual coordinate representation of said specific instance, said representation including specific content sections and associated boundaries;

instantiating said non-visual feedback parameters in one or more user input/output devices such that the requestor recognizes, non-visually, specific content sections and associated boundaries of said graphical image.

26. A method of non-visually representing graphical images by mapping included content sections and associated boundaries to non-visual feedback parameters, said image comprising a multiplicity of content sections and associated boundaries, as per claim 25, wherein said physically recognized feedback parameters comprise tactile feedback for boundaries and specific sounds for content locations.