System and method for maintaining accuracy of voice recognition

• US 10,674,303 B2
• Filed: 09/27/2018
• Issued: 06/02/2020
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Lindahl, Aram M.
• Primary Examiner(s): Chin, Vivian C
• Assistant Examiner(s): Fahnert, Friedrich
• Application Number:
  US16/144,851
• Publication Number:
  US 20190124462A1
• Time in Patent Office: 614 Days
• Field of Search: 381 18, 381 57, 381 731, 381300, 381303, 381307
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04R 2227/009 Signal processing in [PA] systems to enhance the speech intelligibility
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 2430/01 Aspects of volume control, not necessarily automatic, in sound systems
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone

Abstract

Method and system for maintaining accuracy of voice recognition are described herein. The audio system reproducing sound using a loudspeaker array that is housed in a loudspeaker cabinet may selection from a number of sound rendering modes and changing the selected sound rendering mode based on the current playback volume set on the audio system. The sound rendering modes include at least one of: a number of free space modes and a number of complex modes. Other aspects are also described and claimed.

Claims

1. A method for maintaining accuracy of voice recognition in an audio system comprising:

receiving a plurality of input audio channels of a piece of sound program content that is to be converted into sound by a loudspeaker array housed in a loudspeaker cabinet, wherein the audio system includes the loudspeaker array, wherein the loudspeaker array includes a plurality of loudspeaker drivers;

determining a current playback volume set on the audio system for playback of the piece of sound program content; and

selecting one of a plurality of sound rendering modes based on the current playback volume, and changing the selected sound rendering mode based on changes in the current playback volume,

wherein the plurality of sound rendering modes include a plurality of free space modes, wherein the plurality of free space modes includes at least a first volume free space mode, a second volume free space mode, and a third volume free space mode,

wherein, in the first volume free space mode, the loudspeaker array produces audio that is played consistently around the audio system,

wherein, in the second volume free space mode, the loudspeaker array produces a focused sound beam that is aimed at a listening position, wherein to produce the focused sound beam, the loudspeaker drivers in the loudspeaker array that are positioned away from the listening position emit lower energy than the loudspeaker drivers positioned towards the listening position, and

wherein in the third volume free space mode, the loudspeaker array produces a focused sound beam aimed at the listening position while the loudspeaker drivers that are positioned away from the listening position emit higher energy than they do in the second volume free space mode.

2. The method of claim 1, wherein in each of the plurality of free space modes, the loudspeaker array produces sound beams having i) an omni-directional pattern that includes a sum of two or more of the plurality of input audio channels, superimposed with ii) a directional pattern that has a plurality of lobes each lobe containing a difference of the two or more input audio channels.

3. The method of claim 1, wherein selecting one of a plurality of sound rendering modes based on the current playback volume includes:

comparing the current playback volume to a first volume threshold and a second volume threshold;

wherein the first volume threshold is lower than the second volume threshold, wherein selecting the first volume free space mode when the current playback volume is lower than the first volume threshold, selecting the third volume free space mode when the current playback volume is higher than the first volume threshold and lower than the second volume threshold, and selecting the second volume free space mode when the current playback volume is higher than the second volume threshold.

4. The method of claim 1, further comprising:

determining a dynamic measurement of content energy of the piece of sound program content that is to be converted into sound by the loudspeaker array, and

changing the selected sound rendering mode based on the dynamic measurement of content energy of the piece of sound program content.

5. The method of claim 1, further comprising:

determining a residual echo in acoustic signals received by at least one of a plurality of microphones included in the audio system based on the sound being played back by the loudspeaker array; and

changing the selected sound rendering mode based on the residual echo.

6. The method of claim 1, further comprising:

detecting a location of a speaker using at least one of a visual sensor or an audio sensor; and

generating a microphone beamformer in a direction of the location of the speaker to capture the speaker'"'"'s speech.

7. A method for maintaining accuracy of voice recognition in an audio system comprising:

receiving a plurality of input audio channels of a piece of sound program content that is to be converted into sound by a loudspeaker array housed in a loudspeaker cabinet, wherein the audio system includes the loudspeaker array, wherein the loudspeaker array includes a plurality of loudspeaker drivers;

determining a current playback volume set on the audio system for playback the piece of sound program content; and

selecting one of a plurality of sound rendering modes based on the current playback volume, and changing the selected sound rendering mode based on changes the current playback volume,

wherein the plurality of sound rendering modes include a plurality of complex modes,

wherein, in the plurality of complex modes, the loudspeaker array produces i) a sound beam having a direct content pattern that is aimed at a listening position, superimposed with ii) at least one sound beam having an ambient content pattern that is aimed away from the listening position,

wherein the plurality of complex modes includes at least a first volume complex mode, a second volume complex mode, and a third volume complex mode,

wherein the at least one sound beam having the ambient content pattern in the first volume complex mode emits higher energy than the at least one sound beam having the ambient content pattern in the second volume complex mode, and

in the third volume complex mode, the at least one sound beam having the ambient content pattern (i) emits lower energy than does the at least one sound beam having the ambient content pattern in the first volume complex mode, and (ii) emits higher energy than does the least one sound beam having the ambient content pattern in the second volume complex mode.

8. The method of claim 7, wherein the energy is echo coupling energy on at least one microphone in the same housing as the loudspeaker array.

9. The method of claim 7, wherein selecting one of a plurality of sound rendering modes based on the current playback volume includes:

comparing the current playback volume to a first volume threshold and a second volume threshold, wherein the first volume threshold is lower than the second volume threshold,

wherein

selecting the first volume complex mode when the current playback volume is lower than the first volume threshold,

selecting the third volume complex mode when the current playback volume is higher than the first volume threshold and lower than the second volume threshold, and

selecting the second volume complex mode when the current playback volume is higher than the second volume threshold.

10. The method of claim 7, further comprising:

determining a dynamic measurement of content energy of the piece of sound program content that is to be converted into sound by the loudspeaker array, and

changing the selected sound rendering mode based on the dynamic measurement of content energy of the piece of sound program content.

11. The method of claim 7, further comprising:

determining a residual echo in acoustic signals received by at least one of a plurality of microphones included in the audio system based on the sound being played back by the loudspeaker array; and

changing the selected sound rendering mode based on the residual echo.

12. The method of claim 7, further comprising:

detecting a location of a speaker using at least one of a visual sensor or an audio sensor; and

generating a microphone beamformer in a direction of the location of the speaker to capture the speaker'"'"'s speech.

13. The method of claim 7, wherein the plurality of sound rendering modes of operation further include a plurality of free space modes,

wherein the plurality of free space modes includes at least a first volume free space mode and a second volume free space mode,

wherein, in the first volume free space mode, the loudspeaker array produces audio that is played consistently around the audio system, and

wherein, in the second volume free space mode, the loudspeaker array produces a focused sound beam that is aimed at the listening position, wherein to produce the focused sound beam, the loudspeaker drivers positioned away from the listening position emit lower energy than the loudspeaker drivers positioned towards the listening position.

14. The method claim 13, wherein the plurality of free space modes include a third volume free space mode, wherein, in the third volume free space mode, the loudspeaker array produces the focused sound beam aimed at the listening position, wherein the loudspeaker drivers in the loudspeaker array positioned away from the listening position in the third volume free space mode emit higher energy than the loudspeaker drivers positioned away from the listening position in the second volume free space mode.

15. The method of claim 13, wherein a decision to select the one of the plurality of sound rendering modes is further based on whether the audio system is located within a predetermined distance from an acoustically reflective surface, wherein

when the audio system is not located within the predetermined distance from an acoustically reflective surface, one of the plurality of free space modes is selected, and

when the audio system is located within a predetermined distance from an acoustically reflective surface, one of the plurality of complex modes is selected.

16. An audio system for maintaining accuracy of voice recognition comprising:

a plurality of microphones to receive acoustic signals, wherein the acoustic signals include a speaker'"'"'s speech;

a loudspeaker cabinet, having integrated therein a loudspeaker array that includes a plurality of loudspeaker drivers and a plurality of audio amplifiers;

a rendering processor to receive a plurality of input audio channels of a piece of sound program content that is to be converted into sound by the loudspeaker array, the rendering processor having outputs that are coupled to inputs of the plurality of audio amplifiers, the rendering processor having a plurality of sound rendering modes of operation that include a plurality of complex modes,

wherein, in the plurality of complex modes, the loudspeaker array produces i) a sound beam having a direct content pattern that is aimed at a listening position, superimposed with at least one sound beam having an ambient content pattern that is aimed away from the listening position; and

a decision logic

to determine a current playback volume set on the audio system for playback the piece of sound program content, and

to select one of a plurality of sound rendering modes based on the current playback volume, and to change the selected sound rendering mode based on changes the current playback volume,

wherein the plurality of complex modes includes at least a first volume complex modes, a second volume complex mode, and a third volume complex mode,

wherein the at least one sound beam having the ambient content pattern in the first volume complex mode emits higher energy than the at least one sound beam having the ambient content pattern in the second volume complex mode, and

in the third volume complex mode, the at least one sound beam having the ambient content pattern emits lower energy than does the at least one sound beam having the ambient content pattern in the first volume complex mode, and emits higher energy than does the at least one sound beam having the ambient content pattern in the second volume complex mode.

17. The audio system of claim 16, wherein the energy is echo coupling energy on at least one microphone in the same housing as the loudspeaker array.