Gesture controlled audio user interface

• US 9,563,278 B2
• Filed: 10/30/2012
• Issued: 02/07/2017
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Xiang, Pei, Nelson, Hui-ya Liao
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): LUI, DONNA V
• Application Number:
  US13/664,281
• Publication Number:
  US 20130154930A1
• Time in Patent Office: 1,561 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A user interface, methods and article of manufacture each for selecting an audio cue presented in three-dimensional (3D) space are disclosed. The audio cues are audibly perceivable in a space about a user, where each of the audio cues may be perceived by the user as a directional sound at a distinct location from other audio cues in the space. Selection of a specific audio cue is made based on one or more user gestures. A portable electronic device may be configured to present the audio cues perceived by a user and detect certain user gestures to select audio cues. The audio cue selection can be used to control operation of the portable device and/or other associated devices.

Claims

1. An apparatus, comprising:

at least one close-range touch sensor, on an edge of the apparatus, configured to detect which part of a periphery of the apparatus is being gripped and touched by a user, detect a user grasp of the apparatus that corresponds to a user'"'"'s selection of a particular audio cue from among a plurality of audio cues that are audibly perceivable in space about the user, and output a detection of the user grasp; and

a processor configured to generate the plurality of audio cues, each of the audio cues generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space, the processor also being configured to determine user touches at certain locations of the apparatus based on the output of the at least one close-range touch sensor indicating which part of the periphery of the apparatus is being gripped and touched by the user, the processor also being configured to match the output of the at least one close-range touch sensor to one or more of the plurality of audio cues.

2. The apparatus of claim 1, wherein the processor is further configured to match a direction of user movement to one or more of the audio cues.

3. The apparatus of claim 1, wherein the processor is further configured to produce an audio feedback cue based on user movement detected by a user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to a direction of the user movement.

4. The apparatus of claim 1, wherein the apparatus includes a touch screen and the processor is further configured to detect a directional swipe across the touch screen.

5. The apparatus of claim 1, further comprising one or more orientation sensors configured to detect an orientation of the apparatus.

6. The apparatus of claim 5, wherein the processor is further configured to determine an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus, based on output from the one or more orientation sensors.

7. The apparatus of claim 1, wherein the at least one close-range touch sensor is at least one ultrasonic transducer.

8. The apparatus of claim 1, further comprising a plurality of microphones producing microphone signals in response to an ultrasonic signal, wherein the processor is further configured to detect user movement based on the microphone signals, and wherein the processor is also configured to determine an orientation of the apparatus based on the microphone signals.

9. A method, comprising:

detecting, by at least one close-range touch sensor on an edge of an electronic device, which part of a periphery of the electronic device is being gripped and touched by a user;

detecting, by the at least one close-range touch sensor, a user grasp of the electronic device that corresponds to a user'"'"'s selection of a particular audio cue from among a plurality of audio cues that are audibly perceivable in space about the user;

providing, by the at least one close-range touch sensor, an output of a detection of the user grasp;

generating the plurality of audio cues, each of the audio cues being generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

determining user touches at certain locations of the electronic device based on the output of the at least one close-range touch sensor indicating which part of the periphery of the electronic device is being gripped and touched by the user; and

matching the output of the at least one close-range touch sensor to one or more of the plurality of audio cues.

10. The method of claim 9, further comprising matching a direction of user movement to one or more of the audio cues.

11. The method of claim 9, further comprising producing an audio feedback cue based on user movement detected by a user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to a direction of the user movement.

12. The method of claim 9, wherein the electronic device includes a touch screen, and further comprising detecting a directional swipe across the touch screen.

13. The method of claim 9, wherein the electronic device comprises one or more orientation sensors configured to detect an orientation of the electronic device.

14. The method of claim 13, further comprising determining an absolute direction of a user directional swipe, irrespective of the orientation of the electronic device, based on output from the one or more orientation sensors.

15. The method of claim 9, wherein the at least one close-range touch sensor is at least one ultrasonic transducer.

16. The method of claim 9, wherein the electronic device comprises a plurality of microphones producing microphone signals in response to an ultrasonic signal, and further comprising:

detecting user movement based on the microphone signals; and

determining an orientation of the electronic device based on the microphone signals.

17. A non-transitory computer-readable medium embodying a set of instructions executable by one or more processors, comprising:

code for causing at least one close-range touch sensor on an edge of an electronic device to detect which part of a periphery of the electronic device is being gripped and touched by a user;

code for causing the at least one close-range touch sensor to detect a user grasp of the electronic device that corresponds to a user'"'"'s selection of a particular audio cue from among a plurality of audio cues that are audibly perceivable in space about the use′

;

code for causing the at least one close-range touch sensor to provide an output of a detection of the user grasp;

code for generating the plurality of audio cues, each of the audio cues being generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

code for determining user touches at certain locations of the electronic device based on the output of the at least one close-range touch sensor indicating which part of the periphery of the electronic device is being gripped and touched by the user; and

code for matching the output of the at least one-close range touch sensor to one or more of the plurality of audio cues.

18. The non-transitory computer-readable medium of claim 17, further comprising code for matching a direction of user movement to one or more of the audio cues.

19. The non-transitory computer-readable medium of claim 17, further comprising code for producing an audio feedback cue based on user movement detected by a user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to a direction of the user movement.

20. The non-transitory computer-readable medium of claim 17, wherein the electronic device includes a touch screen, and further comprising code for detecting a directional swipe across the touch screen.

21. The non-transitory computer-readable medium of claim 17, wherein the electronic device comprises one or more orientation sensors configured to detect an orientation of the electronic device.

22. The non-transitory computer-readable medium of claim 21, further comprising code for determining an absolute direction of a user directional swipe, irrespective of the orientation of the electronic device, based on the output from the one or more orientation sensors.

23. The non-transitory computer-readable medium of claim 17, wherein the at least one close-range touch sensor is at least one ultrasonic transducer.

24. The non-transitory computer-readable medium of claim 17, wherein the electronic device comprises a plurality of microphones producing microphone signals in response to an ultrasonic signal, and further comprising:

code for detecting user movement based on the microphone signals; and

code for determining an orientation of the electronic device based on the microphone signals.

25. An apparatus, comprising:

means for detecting which part of a periphery of the apparatus is being gripped and touched by a user;

means for detecting a user grasp of the apparatus that corresponds to a user'"'"'s selection of a particular audio cue from among a plurality of audio cues that are audibly perceivable in space about the user;

means for providing an output of a detection of the user grasp;

means for generating the plurality of audio cues, each of the audio cues being generated so as to be perceived by the user as a directional sound at a distinct location from other audio cues in the space;

means for determining user touches at certain locations of the apparatus based on the output of the detection of the user grasp indicating which part of the periphery of the apparatus is being gripped and touched by the user; and

means for matching the output of the detection of the user grasp to one or more of the plurality of audio cues.

26. The apparatus of claim 25, further comprising means for matching a direction of user movement to one or more of the audio cues.

27. The apparatus of claim 25, further comprising means for producing an audio feedback cue based on user movement detected by a user interface, the audio feedback cue being audibly perceivable in the space about the user so as to correspond to a direction of the user movement.

28. The apparatus of claim 25, further comprising means for detecting a directional swipe across a touch screen.

29. The apparatus of claim 25, further comprising means for detecting an orientation of the apparatus.

30. The apparatus of claim 29, further comprising means for determining an absolute direction of a user directional swipe, irrespective of the orientation of the apparatus, based on output from the means for detecting the orientation of the apparatus.

31. The apparatus of claim 25, further comprising:

means for producing microphone signals in response to an ultrasonic signal;

means for detecting user movement based on the microphone signals; and

means for determining an orientation of the apparatus based on the microphone signals.