Method of and apparatus for multi-modal information presentation to computer users with dyslexia, reading disabilities or visual impairment

• US 6,324,511 B1
• Filed: 10/01/1998
• Issued: 11/27/2001
• Est. Priority Date: 10/01/1998
• Status: Expired due to Term

• Current Assignee: Creative Technology Ltd.
• Original Assignee: Mindmaker, Inc.
• Inventors: Kiraly, Jozsef, Ridge, Peter M.
• Primary Examiner(s): Korzuch, William R.
• Assistant Examiner(s): Chawan, Vijay B
• Application Number:
  US09/164,897
• Time in Patent Office: 1,153 Days
• Field of Search: 704/260, 704/271, 704/270
• US Class Current: 704/260
• CPC Class Codes: G09B 21/006 using audible presentation of the information

Abstract

A method of providing language assistance to a computer user with dyslexia, reading disabilities or visual impairment by presenting text-based information via multiple media channels. The method of the present invention includes the steps of: accessing a source of text-based data that can originate from a computer text document, displaying text-based data in a text window with a standard font size, displaying the text-based data in another text window with a magnified font size, sequentially highlighting the text-based data in the magnified text window one word at a time; generating synthesized speech signals representative of the highlighted text; rendering the synthesized speech signals audible synchronously with the displaying of the highlighted text; displaying images that correspond to the context of the highlighted text; and generating sound effects that are pertinent to the context of the highlighted text such that text-based information and corresponding graphical and audible information can be perceived simultaneously by the user. This technique effectively provides multiple channels of information to a user. The present invention is particularly useful for displaying text-based information to users having reading disabilities such as dyslexia, or for increasing the entertainment value of viewing a text document. The present invention may be used for assisting users in editing documents and in retrieving information from the World Wide Web.

Claims

1. In a computer system, a method of displaying text-based information so to provide text-based information reading and comprehension assistance to a user, said method comprising the computer implemented steps of:

accessing a source of text-based data;

analyzing the entire said source of text-based data and deriving context information of said entire source;

automatically adjusting visual display and audio parameters based on said context information of said entire source;

displaying a portion of said text-based data in a first text window displayed on a display screen of said computer system;

sequentially marking a section of said portion of said text-based data one section at a time and displaying said section of text-based data in a second text window displayed on said display screen, wherein said first and second text windows are displayed simultaneously;

synchronous with said step of marking, generating synthesized speech signals representative of said section of said text-based data, and rendering said speech signals audible so that text-based information and corresponding audible information are delivered to said user simultaneously;

providing a third window of said display screen for displaying images;

synchronous with said step of marking, displaying an image contextually relevant to said marked section of said text-based data in said third window such that said image can be perceived in conjunction with hearing said speech signals and seeing said section of text-based data; and

generating audible sound effects synchronously with said step of marking, wherein said audible sound effects are contextually relevant to said section of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said section of said text-based data.

2. The method according to claim 1 wherein said step of displaying said section of text-based data further comprises the step of displaying said text-based data with a magnified font size compared to said first text window.

3. The method according to claim 1 wherein first text window is for displaying said text-based data with a standard font size, and wherein said second text window is for displaying said section of text-based data with a magnified font size compared to said standard font size.

4. The method according to claim 3 wherein a size of said magnified font size is adjustable by said user via a graphical user interface (GUI).

5. The method according to claim 1 wherein said step of marking further comprises the step of highlighting said section of said text-based data on said display screen.

6. The method according to claim 1 wherein said step of marking further comprises the step of underlining said section of said text-based data on said display screen.

7. The method according to claim 1 wherein said step of accessing said source of text-based data comprises the step of receiving said text-based data from a clipboard.

8. The method according to claim 1 wherein said step of accessing said source of text-based data comprises the step of receiving said text-based data from the World Wide Web.

9. The method according to claim 1 wherein voice parameters of said synthesized speech are user-adjustable via a graphical user interface (GUI).

10. The method according to claim 9 wherein said voice parameters include speed, pitch, voice type, volume, echo, balance and inverse echo.

11. The method according to claim 1 wherein voice parameters of said synthesized speech vary according to a context of said section of said text-based data.

12. The method according to claim 1 wherein said synthesized speech signals are selectively rendered from one of a plurality of speakers of said computer system according to a context of said section of said text-based data such that said synthesized speech signals are perceived by said user as directional.

13. The method according to claim 1 wherein said audible sound effects are perceived as three-dimensional by said user.

14. In a computer system, a method of displaying text-based information so to provide text-based information reading and comprehension assistance to a user, said method comprising the computer implemented steps of:

accessing a source of text-based data;

analyzing the entire said source of text-based data and deriving context information of said entire source;

automatically adjusting visual display and audio parameters based on said context information of said entire source;

displaying text-based data in a first text window displayed on a display screen of said computer system wherein said text-based data is displayed with a pre-selected font size;

displaying said text-based data in a second text window displayed on said display screen of said computer system, wherein said text-based data is displayed with a magnified font size compared to said pre-selected font size, and wherein said first text window and said second text window are displayed simultaneously;

sequentially highlighting a word of said text-based data in said second text window one word at a time;

synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;

providing a third window of said display screen for displaying images;

synchronous with said step of highlighting, displaying in said third window an image contextually relevant to said highlighted word such that said image is perceived by said user in conjunction with hearing said synthesized speech signals and viewing said section of said text-based data; and

generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said section of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.

15. The method according to claim 14 wherein a size of said magnified font size is adjustable by said user via a graphical user interface (GUI).

16. The method according to claim 14 wherein a color of said highlighted text-based data is adjustable by said user via a graphical user interface (GUI).

17. The method according to claim 14 wherein voice parameters of said synthesized speech signals are user-adjustable via a graphical user interface (GUI), and wherein said voice parameters include speed, pitch, voice type, volume, echo, balance and inverse echo.

18. The method according to claim 14 wherein voice parameters of said synthesized speech signals vary according to a context of said highlighted word of said text-based data.

19. The method according to claim 14 wherein said synthesized speech signals are selectively rendered from one of a plurality of speakers of said computer system according to a context of said highlighted word such that said synthesized speech signals are perceived by said user as directional.

20. The method according to claim 14 wherein said audible sound effects are perceived as three-dimensional by said user.

21. The method according to claim 14 wherein said step of accessing said source of text-based data further comprises the step of receiving said text-based data from a clipboard.

22. The method according to claim 14 wherein said step of accessing said source of text-based data further comprises the step of receiving said text-based data from the World Wide Web.

23. A computer-usable medium having computer-readable program code embodied therein for causing a computer to perform a method of displaying text-based information to a user with dyslexia, reading disabilities or vision impairment so to provide text-based information reading and comprehension assistance to said user, said method comprising the steps of:

accessing a source of text-based data;

analyzing the entire said source of text-based data and deriving context information of said entire source;

automatically adjusting visual display and audio parameters based on said context information of said entire source;

displaying text-based data in a first text window displayed on a display screen of said computer system, wherein said text-based data is displayed with a pre-selected font size;

displaying said text-based data in a second text window displayed on said display screen of said computer system, wherein said text-based data is displayed with a magnified font size compared to said pre-selected font size, and wherein said first text window and said second text window are displayed simultaneously;

sequentially highlighting a word of said text-based data in said second text window one word at a time;

synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;

providing a third window of said display screen for displaying images;

synchronous with said step of highlighting, displaying in said third window an image contextually relevant to said highlighted word such that said image can be perceived in conjunction with hearing said speech signals and seeing said section of text-based data; and

generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said section of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.

24. The computer-usable medium according to claim 23 wherein voice parameters of said synthesized speech signals are user-adjustable and wherein said voice parameters include speed, pitch, voice-type, volume, echo, balance and inverse echo.

25. The computer-usable medium according to claim 23 wherein voice parameters of said synthesized speech signals vary according to a context of said highlighted word of said text-based data.

26. The computer-usable medium according to claim 23 wherein said synthesized speech signals are selectively rendered from one of a plurality of speakers of said computer system according to a context of said highlighted word such that said synthesized speech signals are perceived by said user as directional.

27. The computer-usable medium according to claim 23 wherein said audible sound effects are perceived as three-dimensional by said user.

28. The computer-usable medium according to claim 23 wherein said step of accessing said source of text-based data further comprises the step of receiving said text-based data from a clipboard.

29. The computer-usable medium according to claim 23 wherein said step of accessing said source of text-based data further comprises the step of receiving said text-based data from the World Wide Web.

30. A computer-usable medium having computer-readable program code embodied therein for causing a computer to perform a method of displaying text-based information to a user with dyslexia, reading disabilities or vision impairment so to provide text-based information reading and comprehension assistance to said user, said method comprising the steps of:

accessing a source of text-based data;

analyzing the entire said source of text-based data and deriving context information of said entire source;

automatically adjusting visual display and audio parameters based on said context information of said entire source;

displaying text-based data in a text window displayed on a display screen of said computer system, wherein said text-based data is displayed with a pre-selected font size;

sequentially highlighting a word of said text-based data in said text window one word at a time;

synchronous with said step of highlighting, generating synthesized speech signals representative of said highlighted word of said text-based data and rendering said synthesized speech signals audible such that text-based information and corresponding audible information can be perceived simultaneously by said user;

providing a graphics window of said display screen for displaying images;

synchronous with said step of highlighting, displaying in said graphics window an image contextually relevant to said highlighted word of said text-based data such that said image can be perceived by said user in conjunction with hearing said speech signals and seeing said highlighted word of text-based data; and

generating audible sound effects synchronously with said step of highlighting, wherein said audible sound effects are contextually relevant to said highlighted word of said text-based data such that said audible sound effects can be perceived in conjunction with hearing said speech signals and seeing said highlighted word of said text-based data.