Audio-visual user interface controls

• US 5,801,692 A
• Filed: 11/30/1995
• Issued: 09/01/1998
• Est. Priority Date: 11/30/1995
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Muzio, Simon C., Omel, Randall R., Linnett, Barry J.
• Primary Examiner(s): KATBAB, ABDOLLAH
• Application Number:
  US08/564,977
• Time in Patent Office: 1,006 Days
• Field of Search: 395/333, 395/334, 395/339, 395/348, 395/977, 395/978, 345/333, 345/334, 345/339, 345/348, 345/349, 345/352, 345/354, 345/977, 345/978
• US Class Current: 715/764
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting

Abstract

Audio-visual user interface controls provide audio and a visual feedback in response to user input events in a graphical user interface environment of a computer. A variety of audio and visual responses of a control assist the user in understanding its operation.

Claims

1. A method for providing an interactive user interface control in a graphical user interface of a programmed computer system, including a pointing device, a display device, and an audio device, the method comprising:

displaying a control upon which a user positions a cursor to enter input;

the control having a set of predefined user input events, wherein the set of predefined user input events includes a user input event that properly actuates the control and a plurality of user input events that do not properly actuate the control;

the control having a set of visual and audio responses that are played in response to the set of predefined user input events, wherein the set of visual and audio responses provides graphical and audio feedback to indicate to the user when the user is properly actuating the control and when the user is improperly actuating the control;

monitoring position of a cursor associated with the pointing device or state of the pointing device to detect the set of predefined user input events;

in response to detecting a user input event that properly actuates the control while the control is active, simultaneously displaying a corresponding visual response in the control and playing a corresponding audio response from memory of the programmed computer system to indicate to the user that the user is properly positioning the cursor on the control; and

in response to detecting a user input event that does not properly actuate the control while the control is active, displaying a corresponding visual response in the control or playing a corresponding audio response from memory of the programmed computer system to indicate to the user that the user is not properly positioning the cursor on the control.

2. The method of claim 1 further including

displaying a first family of controls having a first set of common visual and audio responses corresponding to a first set of predefined user input events;

displaying a second family of controls having a second set of common visual and audio responses corresponding to a second set of predefined user input events, wherein the audio responses in the first set of common visual and audio responses are different from the audio responses in the second set of common visual and audio responses;

monitoring position and state of a pointing device to detect the first and second set of predefined user input events;

in response to detecting a predefined event in the first set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the first set from memory of the programmed computer system; and

in response to detecting a predefined event in the second set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the second set from memory of the programmed computer system.

3. The method of claim 1 further including

displaying a first family of controls having a first set of common visual and audio responses corresponding to a first set of predefined user input events;

displaying a second family of controls having a second set of common visual and audio responses corresponding to a second set of predefined user input events, wherein the visual responses in the first set of common visual and audio responses are different from the visual responses in the second set of common visual and audio responses;

monitoring position and state of a pointing device to detect the first and second set of predefined user input events;

in response to detecting a predefined event in the first set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the first set from memory of the programmed computer system; and

in response to detecting a predefined event in the second set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the second set from memory of the programmed computer system.

4. The method of claim 1 further including

displaying a first family of controls having a first set of common visual and audio responses corresponding to a first set of predefined user input events;

displaying a second family of controls having a second set of common visual and audio responses corresponding to a second set of predefined user input events, wherein the visual and audio responses in the first set of common visual and audio responses are different from the visual and audio responses in the second set of common visual and audio responses;

monitoring position and state of a pointing device to detect the first and second set of predefined user input events;

in response to detecting a predefined event in the first set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the first set from memory of the programmed computer system; and

in response to detecting a predefined event in the second set of predefined user input events, simultaneously displaying a corresponding visual response and playing a corresponding audio response of the second set from memory of the programmed computer system.

5. The method of claim 4 wherein the first set of common visual and audio responses include common animated and audio feedback played simultaneously to indicate proper actuation of the controls in the first family in response to a user input event that causes the proper actuation of the controls and further include common visual and audio feedback played simultaneously to indicate improper actuation of the controls in the first family in response to a user input event that causes the improper actuation of the controls;

and

the second set of common visual and audio responses include common animated and audio feedback played simultaneously to indicate proper actuation of the controls in the second family in response to a user input event that causes the proper actuation of the controls and further include common visual and audio feedback played simultaneously to indicate improper actuation of the controls in the second family in response to a user input event that causes the improper actuation of the controls.

6. The method of claim 1 wherein a visual response in the visual responses is comprised of a sequence of images from an animation in an animation file.

7. The method of claim 1 wherein a visual response in the visual responses is comprised of frames of animation in an animation file.

8. The method of claim 1 wherein an audio response in the audio responses is comprised of a wave file.

9. The method of claim 1 wherein an audio response in the audio responses is comprised of more than one wave file.

10. The method of claim 1 wherein a visual and an associated audio response in the visual and audio responses are comprised of an animation frame and an associated sound frame.

11. The method of claim 1 wherein the control emulates a push button having an outer boundary and an inner section defined by an inner boundary, and the displaying a corresponding visual response includes the step of:

shifting the inner section by a predefined number of pixels in response to detecting a predefined input event;

and the playing a corresponding audio response includes playing a button down sound.

12. The method of claim 1 wherein the control emulates a push button, and the displaying a corresponding visual response includes the step of:

displaying a sequence of animation frames emulating depressing the push button;

and the playing a corresponding audio response includes playing a button down sound.

13. The method of claim 1 further including:

changing an image associated with the cursor to a new image from an original cursor image in response to detecting that the cursor is positioned over the control on a display of the display device;

displaying the new image on the display of the display device for as long as the cursor is positioned over the control; and

changing the image associated with the cursor to the original cursor image in response to detecting that the cursor is no longer positioned over the control on the display of the display device.

14. The method of claim 1 further including:

in response to detecting a first predefined user input event corresponding to actuating a button of the pointing device while the cursor is positioned over the control, playing a first combined animated and audio response to reinforce to the user that the user is actuating the control; and

in response to detecting a second predefined user input event corresponding to releasing the button of the pointing device while the cursor is positioned over the control, playing a second combined animated and audio response which is different from the first audio response to reinforce to the user that the user is actuating the control.

15. The method of claim 1 further including:

in response to detecting a first predefined user input event corresponding to actuating a button of the pointing device while the cursor is positioned over the control in an inactive state, playing a first audio response to indicate that the control is inactive; and

in response to detecting the first predefined user input event while the cursor is positioned over the control in an active state, playing a visual response and a second audio response which is different from the first audio response to indicate that the control is active.

16. A computer readable medium having instructions for performing the steps of claim 1.

17. A method for providing an interactive user interface control in a graphical user interface of a programmed computer system, including a pointing device, a display device, and an audio device, the method comprising:

displaying a control upon which a user positions a cursor to enter input, the control having a boundary defining a two dimensional space occupied by the control on the display device;

associating a set of predefined user input events with a position of a cursor and state of the pointing device relative to the boundary of the control, wherein the set of predefined user input events includes a user input event that properly actuates the control and a plurality of user input events that do not properly actuate the control;

associating a set of visual and audio responses with the control and with the set of predefined user input events, wherein the set of visual and audio responses provides a plurality of graphical and audio feedback to indicate to the user when the user is properly actuating the control and when the user is improperly actuating the control;

monitoring the position of a cursor on a display of the display device as positioned by a user with the pointing device, and monitoring the state of the pointing device to detect the set of predefined user input events;

in response to detecting the user input event that properly actuates the control while the control is active, simultaneously displaying a corresponding visual response in the control and playing a corresponding audio response to indicate to the user that the user is properly positioning the cursor on the control; and

in response to detecting a user input event that does not properly actuate the control while the control is active, displaying a corresponding visual response in the control to indicate to the user that the user is not properly positioning the cursor on the control.

18. The method of claim 17 further including:

in response to detecting a predefined user input event from the set of predefined user input events, displaying a sequence of frames of animation frames.

19. The method of claim 17 further including:

in response to detecting a predefined user input event corresponding to the cursor passing over the boundary of the control, changing the appearance of the cursor for as long as the cursor is within the boundary of the control; and

changing the image associated with the cursor to the original cursor image in response to detecting that the cursor is no longer positioned over the control on the display of the display device.

20. The method of claim 17 further including:

in response to detecting a predefined user input event corresponding to the cursor passing over the boundary of the control, changing the visual appearance of the control.

21. The method of claim 17 further including:

in response to detecting a predefined user input event corresponding to the cursor passing over the boundary of the control, changing the visual appearance of the control, and changing the appearance of the cursor for as long as the cursor is within the boundary of the control.

22. The method of claim 17 further including:

in response to detecting a predefined user input event corresponding to the cursor passing over the boundary of the control, changing the visual appearance of the control, changing the appearance of the cursor for as long as the cursor is within the boundary of the control, and playing an associated sound with the changing of the visual appearance of a control.

23. The method of claim 17 further including:

in response to detecting a first predefined user input event corresponding to actuating the pointing device down while the cursor is positioned over the control, playing a first audio response; and

in response to detecting a second predefined user input event corresponding to releasing the pointing device while the cursor is positioned over the control, playing a second audio response, which is different from the first audio response.

24. The method of claim 17 further including:

in response to detecting a first predefined user input event corresponding to actuating the pointing device down while the cursor is positioned over the control in an inactive state, playing a first audio response to indicate the control is inactive; and

in response to detecting the first predefined user input while the cursor is in the active state, playing a second audio response, which is different from the first audio response to indicate that the control is active.

25. A user interface control in a programmed computer system, upon which a user positions a cursor to enter input, the control comprising:

control boundary data for defining a boundary of the control in a display, the control boundary data stored in an animation file in memory;

a set of predefined user input events associated with the control, wherein the set of predefined user input events includes a user input event that properly actuates the control and a plurality of user input events that do not properly actuate the control while the control is active;

a set of image files in memory linked to the set of predefined user input events, wherein the image files provide graphical feedback to indicate to the user when the control is properly and improperly actuated while the control is active;

a set of sound wave files in memory linked to the set of predefined user input events, wherein the sound wave files provide aural feedback to indicate to the user when the control is properly and improperly actuated while the control is active;

a procedure for receiving user input messages that report the predefined user input events, for controlling the display of an image from the image files in the boundary of the control in response to a first user input message, and for controlling the playback of a sound wave file in response to the first input message, for controlling the display of a different image from the image files in response to a second input message, and for controlling the playback of a different sound wave file in response to the second input message, where the first message corresponds to the user input event in which the cursor is properly positioned on the control, and the second message corresponds to one of the user input events in which the cursor is not properly positioned on the control.