Method and apparatus for detecting and controlling the orientation of a virtual microphone

• US 9,462,379 B2
• Filed: 07/19/2013
• Issued: 10/04/2016
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Ivanov, Plamen A, Oliveira, Rivanaldo S, Zurek, Robert A
• Primary Examiner(s): TON, DAVID L
• Application Number:
  US13/946,150
• Publication Number:
  US 20140270248A1
• Time in Patent Office: 1,173 Days
• Field of Search: 381/92, 381/313, 381/56
• US Class Current: 1/1
• CPC Class Codes: H04M 1/605 involving control of the receiver volume to provide a dual operational mode at close or far distance from the user
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A method for controlling the orientation of a virtual microphone, which is carried out on an electronic device, includes combining and processing signals from a microphone array to create a virtual microphone; receiving data from a sensor of the electronic device; determining, based on the received data, a mode in which the electronic device is being used; and based on the determined mode, directionally orienting the virtual microphone. Possible use modes include a) a stowed use mode, in which the criterion is the electronic device being substantially enclosed by surrounding material; b) a handset (alternately, private) use mode, in which the criterion is the electronic device being held proximate to a user; and c) a handheld (alternately, speakerphone) use mode, in which the criterion is the electronic device being held away from a user.

Claims

1. An electronic device comprising:

a microphone array comprising a plurality of microphones;

a sensor; and

a processor configured to;

combine and process signals from the microphone array to create a virtual microphone;

determine a use mode of the electronic device;

based on output of the sensor, determine an orientation of the electronic device relative to an orientation of a head of a user; and

adjust an orientation of the virtual microphone based on the determined use mode and on the orientation of the electronic device relative to the orientation of the head of the user.

2. The electronic device of claim 1, wherein:

the sensor is a motion-detecting sensor that detects an orientation motion of the electronic device relative to the user head orientation from a first orientation to a second orientation; and

the processor adjusts the orientation of the virtual microphone by beamforming the microphone array in a direction normal to a longitudinal axis of the electronic device throughout the motion from the first orientation to the second orientation.

3. The electronic device of claim 2, wherein microphone array further comprises:

a first pair and a second pair of microphones, each having respective outputs, wherein the first pair and the second pair of microphones comprise at least three planar microphones of the microphone array;

the processor is further configured to;

combine the first microphone pair output and the second microphone pair output such that the orientation of the microphone array is in a direction normal to a longitudinal axis of the electronic device.

4. The electronic device of claim 1, wherein the processor comprises:

a signal conditioning component having an input that is connected to an output of the microphone array.

5. The electronic device of claim 4, further comprising:

a speech recognition engine that receives conditioned signals from the signal conditioning component.

6. The electronic device of claim 1, wherein the sensor is selected from the group consisting of a proximity sensor, a motion sensor, an optical sensor, and an orientation sensor.

7. The electronic device of claim 1, wherein:

the sensor comprises an orientation sensor that detects a real world coordinate orientation of the electronic device,

wherein when the electronic device is in a handheld use mode, the microphone array uses the orientation sensor to adjust the microphone array orientation to be a direction normal to a longitudinal axis of the electronic device.

8. A method for controlling orientation of a virtual microphone on an electronic device, the method comprising:

combining and processing signals from a microphone array to create the virtual microphone;

determining, according to a predefined criterion, a use mode for the electronic device, the use mode being selected from the group consisting of;

a) a stowed use mode, in which the criterion is the electronic device being substantially enclosed by surrounding material,

b) a handset use mode, in which the criterion is the electronic device being held proximate to a user, and

c) a handheld use mode in which the criterion is the electronic device being held away from a user,

when the device is in the stowed use mode, adjusting the virtual microphone to be omnidirectional;

when the device is in the handset use mode, adjusting the virtual microphone by beamforming the microphone array in a direction normal to a longitudinal axis of the electronic device based on sensed input from a sensor of the electronic device; and

when the device is in the handheld use mode, adjusting the virtual microphone by beamforming the array in a direction based on sensed input from a sensor of the electronic device.

9. The method of claim 8, wherein continuously combining and processing audio signals from a microphone array comprises:

activating a first pair and a second pair of microphones from the microphone array, wherein the first pair and the second pair of microphones comprise at least three planar microphones of the microphone array;

combining a first microphone pair output and a second microphone pair output such that the microphones of the microphone array beamform in a direction normal to a longitudinal axis of the electronic device;

signal conditioning the first microphone pair output and the second microphone pair output; and

providing the signal-conditioned outputs to a speech recognition engine.

10. The method of claim 8, wherein the sensor is selected from a group consisting of a proximity sensor, a motion sensor, an optical sensor, and an orientation sensor.

11. A method for controlling the orientation of a virtual microphone on an electronic device, the method comprising:

combining and processing signals from a microphone array to create the virtual microphone;

receiving real world coordinate orientation data from a sensor of the electronic device;

determining, based on the real world coordinate orientation data, that the electronic device is being used in a handheld use mode; and

based on determining that the electronic device is being used in the handheld use mode, directionally orienting the virtual microphone by beamforming microphones of the microphone array at an angle relative to an earth horizon based on the real world coordinate orientation data.

12. The method of claim 11, comprising:

determining that the electronic device is being used in a stowed use mode; and

orienting the virtual microphone to be omnidirectional.

13. The method of claim 11, comprising:

receiving orientation data from an orientation sensor;

determining that the electronic device is being used in a handset use mode;

detecting, with the orientation sensor, a motion of the electronic device relative to a user'"'"'s head from a first orientation to a second orientation; and

continuously combining and processing audio signals from the microphone array to beamform in a direction normal to a longitudinal axis of the electronic device throughout the motion from the first orientation to the second orientation.

14. The method of claim 13, wherein continuously combining and processing audio signals from the plurality of microphones comprises:

activating a first pair and a second pair of microphones from the microphone array, wherein the first pair and the second pair of microphones comprise at least three planar microphones of the microphone array; and

combining a first microphone pair output and a second microphone pair output such that the microphones of the microphone array beamform in a direction normal to a longitudinal axis of the electronic device.

15. The method of claim 14, further comprising:

signal conditioning the first microphone pair output and the second microphone pair output; and

providing the signal-conditioned outputs to a speech recognition engine.

16. The method of claim 11, wherein the sensor is selected from a group consisting of a proximity sensor, a motion sensor, an optical sensor, and an orientation sensor.

17. The method of claim 11, comprising:

receiving real world coordinate orientation data from the sensor;

determining that the electronic device is being used in a handheld use mode; and

beamforming the microphones of the microphone array at an angle relative to an earth horizon based on the real world coordinate orientation data,

wherein the angle is user-configurable.

18. The method of claim 11, comprising:

determining that the electronic device is being used in a stowed use mode; and

turning off the microphones of the microphone array.