Computer-implemented system and method for enhancing audio to individuals participating in a conversation

• US 8,676,572 B2
• Filed: 03/14/2013
• Issued: 03/18/2014
• Est. Priority Date: 02/28/2003
• Status: Active Grant

• Current Assignee: III Holdings 6, LLC
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Aoki, Paul M., Szymanski, Margaret H., Thornton, James D., Wilson, Daniel H., Woodruff, Allison G.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): BAKER, MATTHEW H
• Application Number:
  US13/802,728
• Publication Number:
  US 20130204616A1
• Time in Patent Office: 369 Days
• Field of Search: 704204-205, 370/260, 370/266, 379202-20601
• US Class Current: 704/206
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 3/564 whereby the feature is a sub-conference
• CPC Class Codes: H04M 3/566 relating to a participants right to speak arrangements for multi-party communication with floor control, e.g. for conferences, H04L65/4038, H04L65/4046, H04L65/4053
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 3/569 using the instant speaker's algorithm speech detection per se G10L25/78

Abstract

A computer-implemented system and method for enhancing audio to individuals participating in a conversation is provided. Audio data for individuals participating in one or more conversations is analyzed. Possible conversational configurations of the individuals are generated based on the audio data, and each possible conversational configuration includes one or more subconfigurations of at least two of the individuals. A probability weight is assigned to each of the subconfigurations and includes a likelihood that the individuals of that subconfiguration are participating in one of the conversations. A probability of each possible conversational configuration is determined by combining the probability weights for the subconfigurations of that possible conversational configuration. The possible conversational configuration with the highest probability is selected as a most probable configuration. The individuals participating in the conversations are determined based on the most probable configuration. Audio for each individual participating in the determined conversations is enhanced.

Claims

1. A computer-implemented system for enhancing audio to individuals participating in a conversation, comprising:

a floor analysis module to analyze audio data for a plurality of individuals participating in one or more conversations, to generate based on the audio data, a plurality of possible conversational configurations of the individuals, each possible conversational configuration comprising one or more pair-wise probabilities of at least two of the individuals, to assign a probability weight to each of the pair-wise probabilities comprising a likelihood that the individuals of that pair-wise probability are participating in one of the conversations, to determine a probability of each possible conversational configuration by combining the probability weights for the pair-wise probabilities of that possible conversational configuration, to select one of the possible conversational configurations with the highest probability as a most probable configuration, to determine the individuals participating in the conversations based on the pair-wise probabilities of the most probable configuration, and to enhance audio for each individual participating in the determined conversations; and

a processor to execute the floor analysis module.

2. A system according to claim 1, wherein the audio is enhanced by at least one of volume adjustment, audio effect, and spatialization control.

3. A system according to claim 1, further comprising:

the floor analysis module to provide an override privilege to at least one individual in one of the pair-wise probabilities of the most probable configuration and to override the audio of all the other individuals in that pair-wise probability of the most probable configuration while the at least one individual speaks.

4. A system according to claim 1, further comprising:

the floor analysis module to calculate a mean of the combined probability weights for the pair-wise probabilities of each possible conversational configuration as the probability of that possible conversational configuration.

5. A system according to claim 1, further comprising:

the floor analysis module to provide to at least one individual who has not yet spoken, at least one of the audio from all the determined conversations and the audio for one or more individuals in at least one of the determined conversations.

6. A system according to claim 1, further comprising:

the floor analysis module to determine at least one of the possible conversational by representing each of the plurality of individuals as a node and to form the pair-wise probabilities by connecting two or more nodes with at least one edge and by combining the pair-wise probabilities as the possible conversational configuration.

7. A system according to claim 1, further comprising:

the floor analysis module to detect audio data from a new individual, to generate new possible conversational configurations based on the audio data from the individuals and the new individual, to determine a new probability to each of the new possible conversational configurations, and to select one of the new possible conversational configurations with the highest new probability.

8. A system according to claim 1, wherein the analysis of the audio data is performed using conversational characteristics shared by the plurality of individuals.

9. A computer-implemented method for enhancing audio to individuals participating in a conversation, comprising:

analyzing audio data for a plurality of individuals participating in one or more conversations;

generating based on the audio data, a plurality of possible conversational configurations of the individuals, each possible conversational configuration comprising one or more pair-wise probabilities of at least two of the individuals;

assigning a probability weight to each of the pair-wise probabilities comprising a likelihood that the individuals of that pair-wise probability are participating in one of the conversations;

determining a probability of each possible conversational configuration by combining the probability weights for the pair-wise probabilities of that possible conversational configuration;

selecting one of the possible conversational configurations with the highest probability as a most probable configuration;

determining the individuals participating in the conversations based on the pair-wise probabilities of the most probable configuration; and

enhancing audio for each individual participating in the determined conversations.

10. A method according to claim 9, wherein the audio is enhanced by at least one of volume adjustment, audio effect, and spatialization control.

11. A method according to claim 9, further comprising:

providing an override privilege to at least one individual in one of the pair-wise probabilities of the most probable configuration; and

overriding the audio of all the other individuals in that pair-wise probability of the most probable configuration while the at least one individual speaks.

12. A method according to claim 9, further comprising:

calculating a mean of the combined probability weights for the pair-wise probabilities of each possible conversational configuration as the probability of that possible conversational configuration.

13. A method according to claim 9, further comprising:

providing to at least one individual who has not yet spoken, at least one of the audio from all the determined conversations and the audio for one or more individuals in at least one of the determined conversations.

14. A method according to claim 9, further comprising:

determining at least one of the possible conversational configurations, comprising;

representing each of the plurality of individuals as a node;

forming the pair-wise probabilities by connecting two or more nodes with at least one edge; and

combining the pair-wise probabilities as the possible conversational configuration.

15. A method according to claim 9, further comprising:

detecting audio data from a new individual;

generating new possible conversational configurations based on the audio data from the individuals and the new individual;

determining a new probability to each of the new possible conversational configurations; and

selecting one of the new possible conversational configurations with the highest new probability.

16. A method according to claim 9, wherein the analysis of the audio data is performed using conversational characteristics shared by the plurality of individuals.

17. A non-transitory computer readable storage medium storing code for executing on a computer system to perform the method according to claim 9.