Graphical user interface having sound effects for operating control elements and dragging objects

• US 20020054176A1
• Filed: 05/14/2001
• Published: 05/09/2002
• Est. Priority Date: 05/08/1998
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ulrich, Robert, Rose, Arlo
• Granted Patent: US 7,568,158 B2
• Time in Patent Office: Days
• US Class Current: 715/864
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: Y10S 715/978 Audio interaction as part of an operator interface

Abstract

Systems and methods for providing an enhanced auditory behavior to a graphical user interface are described. Control elements portrayed by the graphical user interface on a display are associated with at least two states. When transitioning between states, a sound effect specified for that transition can be provided to provide further user or designer customization of the interface appearance. Movement of objects can be accompanied by a repeated sound effect. Characteristics of both sound effects can be easily adjusted in volume, pitch and frequency.

Claims

1. A method for providing a sound effect between a first display position associated with an object drawn on a graphical user interface of a computer system and a second display position associated with said object, the method comprising the steps of:

drawing said object in said first display position of a display space controlled by said graphical user interface;

receiving, at said graphical user interface, a user input invoking movement of said object;

retrieving a sustain sound segment; and

repeatedly reproducing said sustain sound segment until said graphical user interface receives a second user input indicating that said object has reached said second display position.

2. The method of claim 1, further comprising the step of:

panning said reproduced sustain sound segment between speakers as said object moves from said first display position to said second display position.

3. The method of claim 2, wherein said step of panning further comprises the steps of:

varying a volume between said speakers between 50 and 150 percent of a recorded volume depending upon a current display position.

4. The method of claim 1, wherein said step of reproducing further comprises the step of:

reproducing said sustain sound segment at a volume specified for movement of said object.

5. The method of claim 1, wherein said step of reproducing further comprises the step of:

reproducing said sustain sound segment at a pitch specified for movement of said object.

6. The method of claim 1, wherein said step of reproducing further comprises the step of:

reproducing said sustain sound segment after a delay specified for movement of said object.

7. The method of claim 1, further comprising the step of:

retrieving, prior to retrieving said sustain sound segment, an attack sound segment that is reproduced prior to repeatedly reproducing said sustain sound segment.

8. The method of claim 1, further comprising the step of:

retrieving and reproducing, after said second user input is received, a decay sound segment.

9. The method of claim 1, further comprising the step of:

selecting, from within a range of frequencies, a frequency for repeatedly reproducing said sustain sound segment.

10. The method of claim 9, wherein said step of selecting further comprises the step of:

setting said range of frequencies to an envelope of about plus/minus 2.5 percent of an original frequency at which said sustain sound segment was recorded.

11. The method of claim 10, wherein said step of selecting further comprises the step of weighting selection of said frequency from within said envelope.

13. The system of claim 12, further comprising:

at least two speakers for outputting said sound effect; and

means for panning said sound effect between speakers as said object moves from said first display position to said second display position.

14. The system of claim 13, wherein said means for panning further comprises:

means for varying a volume between said speakers between 50 and 150 percent of a recorded volume depending upon a current display position.

15. The system of claim 12, wherein a data structure associated with said sound effect includes a volume parameter specified for output of said sound effect.

16. The system of claim 12, wherein a data structure associated with said sound effect includes a pitch parameter specified for output of said sound effect.

17. The system of claim 12, wherein a data structure associated with said sound effect includes a volume gain parameter specified for output of said sound effect.

18. The system of claim 12, wherein a data structure associated with said sound effect includes an attack segment, a sustain segment and a decay segment.

19. The system of claim 18, further comprising:

means for retrieving, prior to retrieving said sustain sound segment, said attack sound segment that is reproduced prior to repeatedly reproducing said sustain sound segment.

20. The system of claim 19, wherein said means for retrieving further comprises:

means for retrieving and reproducing, after said second display position is reached, said decay sound segment.

21. The system of claim 12, further comprising:

means for selecting, from within a range of frequencies, a frequency for repeatedly reproducing said sound effect.

22. The system of claim 21, wherein said means for selecting further comprises:

means for setting said range of frequencies to an envelope of about plus/minus 2.5 percent of an original frequency at which said sound effect was recorded.

23. The system of claim 22, wherein said means for selecting further comprises:

means for weighting selection of said frequency from within said envelope.

25. The method of claim 24, wherein said output characteristic is frequency.

26. The method of claim 25, wherein said frequency is selected from within an envelope of about plus or minus 2.5 percent of an original, recorded frequency.

27. The method of claim 26, wherein said selection is weighted toward said original, recorded frequency.

12. In a computer having a display, a system for providing a sound effect between a first and a second display position of an object on said display, said system comprising:

an input device for controlling movement of a cursor on said display and for generating a user-controlled output when said cursor is positioned over said control element;

a graphical user interface for rendering said object on said display in said first display position and for receiving said user-controlled output; and

a storage device for storing said sound effect associated with movement of said object, wherein when said graphical user interface receives said user-controlled output, and outputs said sound effect until said object reaches said second display position.

24. In a graphical user interface, a method for providing sound effects therein comprising the steps of:

displaying said an object in a first display state;

receiving a user input associated with a transition to a second display state;

identifying a sound effect associated with said transition using a state table;

varying an output characteristic of said sound effect; and

reproducing said sound effect using said varied output characteristic.

28. A computer-readable medium having at least one data structure from which a sound effect can be produced encoded thereon, said data structure comprising:

a first sound segment for initiating said sound effect;

a second sound segment which is repeatable to sustain said sound effect; and

a third sound segment for decaying said sound effect.