GESTURE CONTROLLED AUDIO USER INTERFACE

• US 20130154930A1
• Filed: 10/30/2012
• Published: 06/20/2013
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Xiang, Pei, Nelson, Hui-ya Liao
• Granted Patent: US 9,563,278 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A user interface, methods and article of manufacture each for selecting an audio cue presented in three-dimensional (3D) space are disclosed. The audio cues are audibly perceivable in a space about a user, where each of the audio cues may be perceived by the user as a directional sound at a distinct location from other audio cues in the space. Selection of a specific audio cue is made based on one or more user gestures. A portable electronic device may be configured to present the audio cues perceived by a user and detect certain user gestures to select audio cues. The audio cue selection can be used to control operation of the portable device and/or other associated devices.

Claims

1. An apparatus, comprising:

a circuit configured to generate a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

a user interface configured to detect user movement relative to the apparatus, made in response to the audio cues; and

a selector configured to select at least one of the audio cues based on the user movement.

2. The apparatus of claim 1, wherein the selector matches a direction of the user movement to one or more of the audio cues.

3. The apparatus of claim 1, wherein the circuit is further configured to produce an audio feedback cue based on the user movement detected by the user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement.

4. The apparatus of claim 1, wherein the apparatus further comprises a touch screen, and the user movement is a directional swipe across the touch screen of the apparatus.

5. The apparatus of claim 1, further comprising one or more sensors configured to detect an orientation of the apparatus.

6. The apparatus of claim 5, further comprising a processor configured to determine an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus, based on output from the sensors.

7. The apparatus of claim 1, wherein the user interface includes one or more ultrasonic transducers configured to detect the user movement.

8. The apparatus of claim 7, wherein in the ultrasonic transducers are located along the periphery of the apparatus.

9. The apparatus of claim 1, further comprising:

a speaker array configured to emit an ultrasonic signal;

a portable device including a plurality of microphones producing microphone signals in response to the ultrasonic signal; and

a processor configured to detect the user movement based on the microphone signals.

10. The apparatus of claim 9, wherein the processor is also configured to determine the orientation of the portable device based on the microphone signals.

11. The apparatus of claim 1, further comprising:

a portable device including a speaker array configured to emit an ultrasonic signal;

a plurality of microphones producing microphone signals in response to the ultrasonic signal; and

a processor configured to detect the user movement based on the microphone signals.

12. The apparatus of claim 11, wherein the processor is also configured to determine the orientation of the portable device based on the microphone signals.

13. An apparatus, comprising:

means for generating a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

means for detecting user movement relative to the apparatus, made in response to the audio cues; and

means for selecting at least one of the audio cues based on the user movement.

14. The apparatus of claim 13, further comprising means for matching a direction of the user movement to one or more of the audio cues.

15. The apparatus of claim 13, further comprising means for producing an audio feedback cue based on the user movement detected by the detecting means, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement.

16. The apparatus of claim 13, wherein the apparatus further comprising means for detecting a directional swipe across a touch screen included in the apparatus.

17. The apparatus of claim 13, further comprising means for detecting an orientation of the apparatus.

18. The apparatus of claim 17, further comprising means for determining an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus.

19. The apparatus of claim 13, further comprising means for detecting the user movement with ultrasonic sensors.

20. The apparatus of claim 13, further comprising:

means for emitting an ultrasonic signal;

means for producing microphone signals in response to the ultrasonic signal; and

means for detecting the user movement based on the microphone signals.

21. The apparatus of claim 20, further comprising means for determining the orientation of a portable device based on the microphone signals.

22. A non-transitory computer-readable medium embodying a set of instructions executable by one or more processors, comprising:

code for generating a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

code for detecting user movement relative to a portable device, made in response to the audio cues; and

code for selecting at least one of the audio cues based on the user movement.

23. The computer-readable medium of claim 22, further comprising code for matching a direction of the user movement to one or more of the audio cues.

24. The computer-readable medium of claim 22, further comprising code for producing an audio feedback cue based on detected user movement, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement.

25. The computer-readable medium of claim 22, further comprising code for determining a directional swipe across a touch screen included in the portable device.

26. The computer-readable medium of claim 22, further comprising code for detecting an orientation of the portable device.

27. The computer-readable medium of claim 26, further comprising code for determining an absolute direction of a user directional swipe, irrespective of the orientation of the portable device.

28. The computer-readable medium of claim 22, further comprising code for detecting the user movement with ultrasonic sensors.

29. The computer-readable medium of claim 22, further comprising:

code for emitting an ultrasonic signal;

code for producing microphone signals in response to the ultrasonic signal; and

code for detecting the user movement based on the microphone signals.

30. The computer-readable medium of claim 29, further comprising code for determining the orientation of the portable device based on the microphone signals.

31. A method of providing a user interface at an electronic device, comprising:

generating, by the electronic device, a plurality of audio cues that are audibly perceivable in a space about a user, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

the electronic device detecting user movement relative to the electronic device, made in response to the audio cues; and

selecting at least one of the audio cues based on the user movement.

32. The method of claim 31, further comprising matching a direction of the user movement to one or more of the audio cues.

33. The method of claim 31, further comprising producing an audio feedback cue based on detected user movement, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to the direction of the user movement.

34. The method of claim 31, further comprising detecting a directional swipe across a touch screen included in the electronic device.

35. The method of claim 31, further comprising detecting an orientation of the electronic device.

36. The method of claim 35, further comprising determining an absolute direction of a user directional swipe, irrespective of the orientation of the electronic device.

37. The method of claim 31, further comprising detecting the user movement with ultrasonic sensors.

38. The method of claim 31, further comprising:

emitting an ultrasonic signal;

producing microphone signals in response to the ultrasonic signal; and

detecting the user movement based on the microphone signals.

39. The method of claim 38, further comprising determining the orientation of the electronic device based on the microphone signals.