Device, Method, and Graphical User Interface for Providing Audiovisual Feedback

• US 20170068430A1
• Filed: 09/29/2015
• Published: 03/09/2017
• Est. Priority Date: 09/08/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Brown, Matthew I., Cieplinski, Avi E.
• Granted Patent: US 10,152,300 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

An electronic device provides data to present a user interface with a plurality of user interface objects, including a control user interface object at a first location. The control user interface object is configured to control a parameter. The device receives an input that corresponds to an interaction with the control user interface object. While receiving the input that corresponds to the interaction with the control user interface object, the device provides data to move the control user interface object, in accordance with the input, from the first location to a second location. The device also provides first sound information to provide a sound output with characteristics that are different from the parameter controlled by the control user interface object and that change with movement of the control user interface object from the first location to the second location.

Claims

1. A method, comprising:

at an electronic device with one or more processors and memory, wherein the device is in communication with a display and an audio system;

providing, to the display, data to present a user interface with a plurality of user interface objects, including a control user interface object at a first location on the display, wherein the control user interface object is configured to control a respective parameter;

receiving a first input that corresponds to a first interaction with the control user interface object on the display; and

while receiving the first input that corresponds to the first interaction with the control user interface object on the display;

providing, to the display, data to move the control user interface object, in accordance with the first input, from the first location on the display to a second location on the display, distinct from the first location on the display; and

providing, to the audio system, first sound information to provide a first sound output with one or more characteristics that are different from the respective parameter controlled by the control user interface object and that change in accordance with movement of the control user interface object from the first location on the display to the second location on the display.

2. The method of claim 1, wherein:

in accordance with a determination that the first input meets first input criteria, the first sound output has a first set of characteristics; and

,

in accordance with a determination that the first input meets second input criteria, the first sound output has a second set of characteristics that are different from the first set of characteristics.

3. The method of claim 1, including:

after responding to the first input, receiving a second input that corresponds to a second interaction with the control user interface object on the display;

in response to and while receiving the second input that corresponds to the second interaction with the control user interface object on the display;

providing, to the display, data to move the control user interface object, in accordance with the second input, from the second location on the display to a third location on the display, distinct from the second location on the display; and

providing, to the audio system, second sound information to provide a second sound output with one or more characteristics that change in accordance with movement of the control user interface object from the second location on the display to the third location on the display.

4. The method of claim 1, wherein the one or more characteristics include a pitch of the first sound output, a volume of the first sound output, and/or a distribution of the first sound output over a plurality of spatial channels.

5. The method of claim 1, wherein:

the audio system is coupled with a plurality of speakers that corresponds to a plurality of spatial channels; and

providing, to the audio system, the first sound information to provide the first sound output includes determining a distribution of the first sound output over the plurality of spatial channels in accordance with a direction of the movement of the control user interface object from the first location on the display to the second location on the display.

6. The method of claim 1, wherein:

the audio system is coupled with a plurality of speakers that corresponds to a plurality of spatial channels; and

providing, to the audio system, the first sound information to provide the first sound output includes determining a distribution of the first sound output over the plurality of spatial channels in accordance with a location of the control user interface object on the display during the movement of the control user interface object from the second location on the display to the third location on the display.

7. The method of claim 1, wherein:

providing, to the audio system, the first sound information to provide the first sound output includes determining a volume of the first sound output in accordance with a speed of the movement of the control user interface object from the first location on the display to the second location on the display.

8. The method of claim 1, wherein:

the control user interface object is a thumb on a slider; and

a pitch of the first sound output changes in accordance with a position of the control user interface object on the slider.

9. The method of claim 1, wherein:

the control user interface object is a thumb on a slider;

the second location on the display is not a terminus of the slider; and

the method includes;

receiving an input that corresponds to a respective interaction with the control user interface object on the display; and

,

in response to receiving the input that corresponds to the respective interaction with the control user interface object on the display;

providing, to the display, data to move the control user interface object, in accordance with the input, to a fourth location on the display, wherein the fourth location on the display is a terminus of the slider; and

providing, to the audio system, sound information to provide a third sound output to indicate that the control user interface object is located at a terminus of the slider, wherein the third sound output is distinct from the first sound output.

10. The method of claim 1, including:

in response to receiving the first input that corresponds to the first interaction with the control user interface object on the display;

providing, to the display, data to move the control user interface object, in accordance with the first input, from the first location on the display to the second location on the display, distinct from the first location on the display, and visually distinguish the control user interface object in accordance with the first input during the movement of the control user interface object from the first location on the display to the second location on the display.

11. An electronic device in communication with a display and an audio system, the device comprising:

one or more processors; and

memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for;

providing, to the display, data to present a user interface with a plurality of user interface objects, including a control user interface object at a first location on the display, wherein the control user interface object is configured to control a respective parameter;

receiving a first input that corresponds to a first interaction with the control user interface object on the display; and

while receiving the first input that corresponds to the first interaction with the control user interface object on the display;

providing, to the display, data to move the control user interface object, in accordance with the first input, from the first location on the display to a second location on the display, distinct from the first location on the display; and

providing, to the audio system, first sound information to provide a first sound output with one or more characteristics that are different from the respective parameter controlled by the control user interface object and that change in accordance with movement of the control user interface object from the first location on the display to the second location on the display.

12. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device that is in communication with a display and an audio system, cause the device to:

provide, to the display, data to present a user interface with a plurality of user interface objects, including a control user interface object at a first location on the display, wherein the control user interface object is configured to control a respective parameter;

receive a first input that corresponds to a first interaction with the control user interface object on the display; and

while receiving the first input that corresponds to the first interaction with the control user interface object on the display;

provide, to the display, data to move the control user interface object, in accordance with the first input, from the first location on the display to a second location on the display, distinct from the first location on the display; and

provide, to the audio system, first sound information to provide a first sound output with one or more characteristics that are different from the respective parameter controlled by the control user interface object and that change in accordance with movement of the control user interface object from the first location on the display to the second location on the display.