Methods, apparatuses and computer program products for manipulating characteristics of audio objects by using directional gestures

• US 9,632,683 B2
• Filed: 11/08/2012
• Issued: 04/25/2017
• Est. Priority Date: 11/08/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Jarvinen, Kari Juhani, Yliaho, Marko Tapani, Virolainen, Jussi Kalevi, Sassi, Antti Heikki Tapio
• Primary Examiner(s): IPAKCHI, MARYAM M
• Application Number:
  US13/672,564
• Publication Number:
  US 20140129937A1
• Time in Patent Office: 1,629 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: G08C 2201/71 Directional beams
• CPC Class Codes: H04B 1/202 by remote control
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/304 For headphones

Abstract

An apparatus for enabling manipulation of one or more audio objects may include a processor and memory storing executable computer program code that cause the apparatus to at least perform operations including detecting at least one audio object and operating a pointer in a selected direction by a user. The computer program code may further cause the apparatus to detect a gesture of the pointer in response to the operating of the pointer in the selected direction. The computer program code may further cause the apparatus to control at least one characteristic of the detected audio object in response to the detected gesture of the pointer or an indication of a selection of the at least one characteristic. Corresponding methods and computer program products are also provided.

Claims

1. A method comprising:

detecting at least one audio object;

operating a pointer in a selected direction, by a user, towards a position of a virtual source of the detected audio object outputting audio to select the virtual source which comprises spatial audio in the selected direction;

detecting a gesture movement of the pointer itself in response to the operating of the pointer in the selected direction towards the position of the virtual source of spatial audio associated with the detected audio object;

controlling, via a processor, at least one characteristic of the detected audio object in the selected direction in response to the detected gesture of the pointer or an indication of a selection of the at least one characteristic; and

manipulating the virtual source of the detected audio object via the operating of the pointer in the selected direction.

2. The method of claim 1, further comprising:

determining an amount in which to manipulate the at least one characteristic in response to detecting a movement of the pointer; and

manipulating the at least one characteristic of the detected audio object by the determined amount, wherein the detected audio object comprises an audio signal.

3. The method of claim 1, wherein detecting the at least one audio object in response to operating the pointer towards the detected audio object and the pointer comprises at least one of a hand, a palm, a finger of the user or a device.

4. The method of claim 3, wherein the device comprises a mobile terminal.

5. The method of claim 2, wherein detecting the movement comprises detecting at least one of a positional change of the pointer, a rotation of the pointer in a clockwise direction or a counter clockwise direction.

6. The method of claim 5, further comprising:

adjusting the characteristic based on at least one of the positional change of the pointer or the rotation of the pointer in the clockwise direction or the counter clockwise direction.

7. The method of claim 6, wherein the pointer is associated with at least one of:

a virtual slider that is movable in at least one dimensional axis in which the pointer is moved in the at least one dimensional axis, or

a virtual knob that is rotatable in the clockwise direction in an instance in which the pointer is rotated in the clockwise direction and the virtual knob is rotatable in the counter clockwise direction in an instance in which the pointer is rotated in the counter clockwise direction.

8. The method of claim 1, wherein:

controlling further comprises at least one of;

adjusting the characteristic of the detected audio object or moving the detected audio object to a new location based in part on the user utilizing the pointer to perform one or more predefined gestures or in response to receipt of an indication of one or more selections indicating to move the detected audio object to the new location, or

changing a distance between the detected audio object and the user based in part on detecting that the pointer is being utilized to perform at least one gesture or based in part on receipt of an indication of a selection to change the distance.

9. The method of claim 3, wherein the characteristic of the at least one detected audio object is predefined in the device by the user.

21. The method of claim 1, wherein prior to controlling the at least one characteristic, the method further comprises:

detecting whether a location of the detected audio object is to be moved to another location based in part on the detected gesture movement.

10. An apparatus comprising:

at least one processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following;

detect at least one audio object;

operate a pointer in a selected direction, by a user, towards a position of a virtual source of the detected audio object outputting audio to select the virtual source which comprises spatial audio in the selected direction;

detect a gesture movement of the pointer itself in response to the operating of the pointer in the selected direction towards the position of the virtual source of spatial audio associated with the detected audio object;

control at least one characteristic of the detected audio object in the selected direction in response to the detected gesture of the pointer or an indication of a selection of the at least one characteristic; and

manipulate the virtual source of the detected audio object via the operating of the pointer in the selected direction.

11. The apparatus of claim 10, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to:

determine an amount in which to manipulate the at least one characteristic in response to detecting a movement of the pointer; and

manipulate the at least one characteristic of the detected audio object by the determined amount, wherein the detected audio object comprises an audio signal.

12. The apparatus of claim 10, wherein detect the at least one audio object in response to operating the pointer towards the detected audio object and the pointer comprises at least one of a hand, a palm, a finger of the user, or the apparatus.

13. The apparatus of claim 12, wherein the apparatus comprises a mobile terminal.

14. The apparatus of claim 11, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to:

perform the detecting of the movement by detecting at least one of a positional change of the pointer, a rotation of the pointer in a clockwise direction or a counter clockwise direction.

15. The apparatus of claim 14, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to:

adjust the characteristic based on at least one of the positional change of the pointer or the rotation of the pointer in the clockwise direction or the counter clockwise direction.

16. The apparatus of claim 15, wherein the pointer is associated with at least one of:

a virtual slider that is movable in at least one dimensional axis in which the pointer is moved in the at least one dimensional axis, or

a virtual knob that is rotatable in the clockwise direction in an instance in which the pointer is rotated in the clockwise direction and the virtual knob is rotatable in the counter clockwise direction in an instance in which the pointer is rotated in the counter clockwise direction.

17. The apparatus of claim 10, wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to:

control the at least one characteristic by at least one of;

adjusting the characteristic of the detected audio object or moving the detected audio object to a new location based in part on the user utilizing the pointer to perform one or more predefined gestures or in response to receipt of an indication of one or more selections indicating to move the detected audio object to the new location, or

changing a distance between the detected audio object and the user based in part on detecting that the pointer is being utilized to perform at least one gesture or based in part on receipt of an indication of a selection to change the distance.

18. The apparatus of claim 12, wherein the characteristic of the detected audio object is predefined in the apparatus by the user.

22. The apparatus of claim 10, wherein prior to control the at least one characteristic, the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to:

detect whether a location of the detected audio object is to be moved to another location based in part on the detected gesture movement.

19. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising:

program code instructions configured to detect at least one audio object;

program code instructions configured to operate a pointer in a selected direction, by a user, towards a position of a virtual source of the detected audio object outputting audio to select the virtual source which comprises spatial audio in the selected direction;

program code instructions configured to detect a gesture movement of the pointer itself in response to the operating of the pointer in the selected direction towards the position of the virtual source of spatial audio associated with the detected audio object;

program code instructions configured to control at least one characteristic of the detected audio object in the selected direction in response to the detected gesture of the pointer or an indication of a selection of the at least one characteristic; and

program code instructions configured to manipulate the virtual source of the detected audio object via the operating of the pointer in the selected direction.

20. The computer program product of claim 19, further comprising:

program code instructions configured to determine an amount in which to manipulate the at least one characteristic in response to detecting a movement of the pointer; and

program code instructions configured to manipulate the at least one characteristic of the detected audio object by the determined amount, wherein the detected audio object comprises an audio signal.

23. The computer program product of claim 19, wherein prior to control the at least one characteristic, the computer program product further comprises:

program code instructions configured to detect whether a location of the detected audio object is to be moved to another location based in part on the detected gesture movement.