Method of acoustically expressing image information

• US 5,715,412 A
• Filed: 12/18/1995
• Issued: 02/03/1998
• Est. Priority Date: 12/16/1994
• Status: Expired due to Term

• Current Assignee: Hitachi Maxell Limited (Hitachi, Ltd.)
• Original Assignee: Hitachi, Ltd.
• Inventors: Aritsuka, Toshiyuki, Takahashi, Toshikazu, Nejime, Yoshito, Hataoka, Nobuo
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): SAX, STEVEN PAUL
• Application Number:
  US08/572,892
• Time in Patent Office: 778 Days
• Field of Search: 395/326, 395/336, 395/337, 395/338, 395/339, 395/340, 395/347
• US Class Current: 715/729
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G09B 21/001 Teaching or communicating with blind persons G09B21/02 - G09B21/06 take precedence
• CPC Class Codes: G09B 21/006 using audible presentation of the information

Abstract

The present invention relates to a method of generating sounds by a graphical user interface (GUI) used on information processing apparatuses and computers and, more particularly, to a method of acoustically expressing image information to enable visually disabled people having difficulty in using GUI to operate apparatuses with GUI, and an apparatus for carrying out the method. A specific sound is assigned before hand to an object displayed in a drawing space on a display screen or the like, and the object is expressed acoustically by generating the sound by a sound generating device for a period of time corresponding to the length of the outline of the object, the travel of the object or the ratio of change in size of the object.

Claims

1. A method of acoustically expressing image information about a shape of an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table relating respective shape attributes of objects with respective timbres of sounds assigned to the respective shape attributes of the objects;

specifying an object displayed on the display screen;

specifying at least one sound assigned to at least one shape attribute of the object referring to the look-up table; and

producing acoustic information about the at least one shape attribute of the object by generating the at least one sound assigned to the shape attribute of the object.

2. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein the attribute of the shape of the object is the length of the outline of the object, and the duration of generation of the sound is controlled according to the length of the outline.

4. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 2, wherein the timbre or the tone of the sound assigned to the shape of the object is determined according to the inclinations of the sections of the outline of the object.

5. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 2, wherein vertices, if any, of the shape of the object are expressed acoustically by interrupting the generation of the sound assigned to the shape of the object at time positions respectively corresponding to the vertices.

6. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 2, wherein vertices, if any, of the shape of the object are expressed acoustically by changing the timbre or the tone of the sound assigned to the shape of the object at time positions respectively corresponding to the vertices.

3. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein the attribute of the shape of the object is the curvature of the outline of the object, and the timbre or the tone of the sound assigned to the shape of the object is controlled according to the curvature of the outline.

7. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein, when the object has a concealed portion, the concealed portion is expressed acoustically by expressing the concealed portion by a sound different from the sound assigned to the shape of the object.

8. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein the duration of generation of the sound assigned to the shape of the object is controlled according to a value indicating a change in the position of the object relative to the display screen, and the shape of the object is expressed acoustically.

9. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 8, wherein the value indicating a change in the position of the object relative to the display screen is the travel of the object in the display screen.

10. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 8, wherein the value indicating a change in the position of the object relative to the display screen is a change in the ratio of the area of the object to that of the display screen.

11. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein the duration of generation of the sound assigned to the shape of the object is controlled, the output sound is controlled according to the gradation of a color representing the object, and the shape of the object is expressed acoustically by acoustic information.

12. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 11, wherein the variation of the output sound assigned to the color is the variation of the output sound in band-pass frequency characteristic.

13. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 11, wherein the output sound assigned to the color is a band noise obtained by filtering white noise by a band-pass filter of a specific band-pass frequency characteristics.

14. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table showing the relation between the shapes of objects and the timbres of sounds assigned to the shapes of the objects;

specifying an object displayed on the display screen;

specifying the sound assigned to the object referring to the look-up table; and

producing acoustic information about the attribute of the shape of the object by generating the sound assigned to the shape of the object in a mode corresponding to the attribute of the shape of the object;

wherein the coordinates of the outline, vertices or the center of gravity of the object in a drawing space are mapped to coordinates indicating positions in an acoustic space, and the object is expressed by localizing the sound assigned to the object at the coordinates in the acoustic space.

15. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 14, wherein the relation between a plurality of overlapping objects in the drawing space is expressed by localizing the sounds respectively assigned to the objects at different positions in an acoustic space, respectively.

16. A method of acoustically expressing image information about an object displayed on a graphical user interface display screen, according to claim 14, wherein the sound assigned to the object is generated when the object is selected on the display screen or when a position indicated by a position indicating means is within an area held by the object.

17. A method of acoustically expressing image information about a shape of an object displayed on a graphical user interface display screen, comprising:

preparing a look-up table relating respective shape attributes of objects with respective timbres of sounds assigned to the respective shape attributes of the objects;

specifying an object displayed on the display screen;

specifying two sounds assigned to at least two shape attributes of the object referring to the look-up table; and

producing acoustic information about the at least two shape attributes of the object by generating the at least two sounds assigned to the shape attributes of the object.