Systems, methods, apparatus, and computer-readable media for gestural manipulation of a sound field

• US 10,448,161 B2
• Filed: 02/25/2013
• Issued: 10/15/2019
• Est. Priority Date: 04/02/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Xiang, Pei, Visser, Erik
• Primary Examiner(s): Gay, Sonia L
• Application Number:
  US13/775,720
• Publication Number:
  US 20130259238A1
• Time in Patent Office: 2,423 Days
• Field of Search: 381 17, 381 18, 381310
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04R 2203/12 Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Gesture-responsive modification of a generated sound field is described.

Claims

1. A method of signal processing, said method comprising:

producing a command in response to a representation of a hand gesture of a user, wherein the hand gesture indicates (i) a change to a sound field near the user and (ii) a direction of where to direct the sound field, wherein the sound field comprises a beam;

producing a filter configuration for a spatially directive filter in response to said command based on a current filter configuration;

applying the spatially directive filter to a multichannel signal; and

driving an array of directionally controllable transducers with the multichannel signal to change the sound field that includes the beam.

2. The method of signal processing according to claim 1, wherein said producing the command comprises selecting the command, based on information from said representation, from among a plurality of commands that includes (A) a command to increase a width of the beam and (B) a command to decrease the width of the beam.

3. A method of signal processing according to claim 1, wherein said producing the command comprises selecting the command, based on information from said representation, from among a plurality of commands that includes (A) a command to change a current direction of the beam to a first direction that is on a first side of the beam and (B) a command to change the current direction of the beam to a second direction that is on a second side of the beam opposite to the first side.

4. A method of signal processing according to claim 1, wherein said producing the filter configuration is based on an indication of at least one among a current direction of the beam and a current width of the beam.

5. A method of signal processing according to claim 1, wherein said producing the filter configuration is based on an indication of a current location of a user.

6. A method of signal processing according to claim 1, wherein said producing the filter configuration comprises selecting the filter configuration, according to said command, from among a plurality of filter configurations.

7. The method of signal processing according to claim 6, wherein a first filter configuration among the plurality of filter configurations describes a different phase relation among output channels of the spatially directive filter than a second filter configuration among the plurality of filter configurations.

8. A method of signal processing according to claim 1, wherein the representation of the hand gesture is based on a sequence of images of a user performing the hand gesture within the sound field.

9. The method of signal processing according to claim 8, wherein said sequence of images includes images of a light pattern projected on a hand of the user.

10. A method of signal processing according to claim 1, wherein the hand gesture includes a lateral movement of a hand of a user.

11. A method of signal processing according to claim 1, wherein the hand gesture includes a grasping motion of a hand of a user.

12. A method of signal processing according to claim 1, wherein the hand gesture includes movement of two hands of a user toward each other.

13. A method of signal processing according to claim 1, wherein the hand gesture includes movement of two hands of a user away from each other.

14. A method of signal processing according to claim 1, wherein the hand gesture includes movement of two hands of a user in a same lateral direction.

44. The method according to claim 1, wherein the applying the spatially directive filter is based on the filter configuration.

48. The method of claim 1, further comprising detecting the hand gesture using a camera.

15. An apparatus for signal processing, said apparatus comprising:

means for producing a command in response to a representation of a hand gesture of a user, wherein the hand gesture indicates (i) a change to a sound field near the user and (ii) a direction of where to direct the sound field, wherein the sound field comprises a beam;

means for producing a filter configuration for a spatially directive filter in response to said command based on a current filter configuration;

means for applying the spatially directive filter to a multichannel signal; and

means for driving an array of directionally controllable transducers with the multichannel signal to change the sound field that includes the beam.

16. The apparatus for signal processing according to claim 15, wherein said means for producing the command comprises means for selecting the command, based on information from said representation, from among a plurality of commands that includes (A) a command to increase a width of the beam and (B) a command to decrease the width of the beam.

17. An apparatus for signal processing according to claim 15, wherein said means for producing the command comprises means for selecting the command, based on information from said representation, from among a plurality of commands that includes (A) a command to change a current direction of the beam to a first direction that is on a first side of the beam and (B) a command to change the current direction of the beam to a second direction that is on a second side of the beam opposite to the first side.

18. An apparatus for signal processing according to claim 15, wherein said means for producing the filter configuration is configured to produce the filter configuration based on an indication of at least one among a current direction of the beam and a current width of the beam.

19. An apparatus for signal processing according to claim 15, wherein said means for producing the filter configuration is configured to produce the filter configuration based on an indication of a current location of a user.

20. An apparatus for signal processing according to claim 15, wherein said means for producing the filter configuration comprises means for selecting the filter configuration, according to said command, from among a plurality of filter configurations.

21. The apparatus for signal processing according to claim 20, wherein a first filter configuration among the plurality of filter configurations describes a different phase relation among output channels of the spatially directive filter than a second filter configuration among the plurality of filter configurations.

22. An apparatus for signal processing according to claim 15, wherein the representation of the hand gesture is based on a sequence of images of a user performing the hand gesture within a sound field.

23. The apparatus for signal processing according to claim 22, wherein said sequence of images includes images of a light pattern projected on a hand of the user.

24. An apparatus for signal processing according to claim 15, wherein the hand gesture includes a lateral movement of a hand of a user.

25. An apparatus for signal processing according to claim 15, wherein the hand gesture includes a grasping motion of a hand of a user.

26. An apparatus for signal processing according to claim 15, wherein the hand gesture includes movement of two hands of a user toward each other.

27. An apparatus for signal processing according to claim 15, wherein the hand gesture includes movement of two hands of a user away from each other.

28. An apparatus for signal processing according to claim 15, wherein the hand gesture includes movement of two hands of a user in a same lateral direction.

45. The apparatus according to claim 15, wherein the means for applying the spatially directive filter is based on the filter configuration.

29. An apparatus for signal processing, said apparatus comprising:

a gesture interpreter configured to produce a command in response to a representation of a hand gesture of a user, wherein the hand gesture indicates (i) a change to a sound field near the user and (ii) a direction of where to direct the sound field, wherein the sound field comprises a beam;

a command interpreter configured to produce a filter configuration for a spatially directive filter in response to said command based on a current filter configuration;

a synthesizer configured to apply the spatially directive filter to a multichannel signal; and

an audio output stage configured to drive an array of directionally controllable transducers with the multichannel signal to change the sound field that includes the beam.

30. The apparatus according to claim 29, wherein said gesture interpreter is configured to produce the command by selecting the command, based on information from said representation, from among a plurality of commands that includes a command to increase a width of the beam and a command to decrease the width of the beam.

31. An apparatus according to claim 29, wherein said gesture interpreter is configured to produce the command by selecting the command, based on information from said representation, from among a plurality of commands that includes (A) a command to change a current direction of the beam to a first direction that is on a first side of the beam and (B) a command to change the current direction of the beam to a second direction that is on a second side of the beam opposite to the first side.

32. An apparatus according to claim 29, wherein said command interpreter is configured to produce the filter configuration based on an indication of at least one among a current direction of the beam and a current width of the beam.

33. An apparatus according to claim 29, wherein said command interpreter is configured to produce the filter configuration based on an indication of a current location of a user.

34. An apparatus according to claim 29, wherein said command interpreter is configured to produce the filter configuration by selecting the filter configuration, according to said command, from among a plurality of filter configurations.

35. The apparatus according to claim 34, wherein a first filter configuration among the plurality of filter configurations describes a different phase relation among output channels of the synthesizer than a second filter configuration among the plurality of filter configurations.

36. An apparatus according to claim 29, wherein said audio output stage is configured to drive the directionally controllable transducers to produce a sound field that includes the beam, and

wherein the representation of the hand gesture is based on a sequence of images of a user performing the hand gesture within the sound field.

37. The apparatus according to claim 36, wherein said sequence of images includes images of a light pattern projected on a hand of the user.

38. An apparatus according to claim 29, wherein the hand gesture includes a lateral movement of a hand of a user.

39. An apparatus according to claim 29, wherein the hand gesture includes a grasping motion of a hand of a user.

40. An apparatus according to claim 29, wherein the hand gesture includes movement of two hands of a user toward each other.

41. An apparatus according to claim 29, wherein the hand gesture includes movement of two hands of a user away from each other.

42. An apparatus according to claim 29, wherein the hand gesture includes movement of two hands of a user in a same lateral direction.

46. The apparatus according to claim 29, wherein the applying the spatially directive filter is based on the filter configuration.

49. The apparatus of claim 29, further comprising a camera configured to detect the representation of the hand gesture.

50. The apparatus of claim 29, wherein the gesture interpreter and the synthesizer are integrated in a processor.

51. The apparatus of claim 29, further comprising the directionally controllable transducers.

52. The apparatus of claim 29, wherein the hand gesture indicates a change to a width of the beam.

53. The apparatus of claim 29, wherein the hand gesture indicates an amplification of the sound field.

54. The apparatus of claim 29, wherein the hand gesture indicates an attenuation of the sound field.

43. A non-transitory computer-readable medium having tangible features that cause a machine reading the features to:

produce a command in response to a representation of a hand gesture of a user, wherein the hand gesture indicates (i) a change to a sound field near the user and (ii) a direction of where to direct the sound field, wherein the sound field comprises a beam;

produce a filter configuration for a spatially directive filter in response to said command based on a current filter configuration;

apply the spatially directive filter to a multichannel signal; and

drive an array of directionally controllable transducers with the multichannel signal to change the sound field that includes the beam.

47. The computer-readable medium according to claim 43, wherein the applying the spatially directive filter is based on the filter configuration.