Computer-Implemented System and Method for Enhancing Audio to Individuals Participating in a Conversation

• US 20130204616A1
• Filed: 03/14/2013
• Published: 08/08/2013
• Est. Priority Date: 02/28/2003
• Status: Active Grant

• Current Assignee: III Holdings 6, LLC (Intellectual Ventures LLC)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Aoki, Paul M., Szymanski, Margaret H., Thornton, James D., Woodruff, Allison G., Wilson, Daniel H.
• Granted Patent: US 8,676,572 B2
• Time in Patent Office: Days
• US Class Current: 704/226
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 3/564 whereby the feature is a sub-conference
• CPC Class Codes: H04M 3/566 relating to a participants right to speak arrangements for multi-party communication with floor control, e.g. for conferences, H04L65/4038, H04L65/4046, H04L65/4053
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 3/569 using the instant speaker's algorithm speech detection per se G10L25/78

Abstract

A computer-implemented system and method for enhancing audio to individuals participating in a conversation is provided. Audio data for individuals participating in one or more conversations is analyzed. Possible conversational configurations of the individuals are generated based on the audio data, and each possible conversational configuration includes one or more subconfigurations of at least two of the individuals. A probability weight is assigned to each of the subconfigurations and includes a likelihood that the individuals of that subconfiguration are participating in one of the conversations. A probability of each possible conversational configuration is determined by combining the probability weights for the subconfigurations of that possible conversational configuration. The possible conversational configuration with the highest probability is selected as a most probable configuration. The individuals participating in the conversations are determined based on the most probable configuration. Audio for each individual participating in the determined conversations is enhanced.

Claims

1. A computer-implemented system for enhancing audio to individuals participating in a conversation, comprising:

an audio data analysis module to analyze audio data for a plurality of individuals participating in one or more conversations;

a conversational configuration module to generate based on the audio data, a plurality of possible conversational configurations of the individuals, each possible conversational configuration comprising one or more subconfigurations of at least two of the individuals;

a weight module to assign a probability weight to each of the subconfigurations comprising a likelihood that the individuals of that subconfiguration are participating in one of the conversations;

a probability module to determine a probability of each possible conversational configuration by combining the probability weights for the subconfigurations of that possible conversational configuration;

a selection module to select one of the possible conversational configurations with the highest probability as a most probable configuration;

a conversation determination module to determine the individuals participating in the conversations based on the subconfigurations of the most probable configuration; and

an enhancement module to enhance audio for each individual participating in the determined conversations.

2. A system according to claim 1, wherein the audio is enhanced by at least one of volume adjustment, audio effect, and spatialization control.

3. A system according to claim 1, further comprising:

a privilege module to provide an override privilege to at least one individual in one of the subconfigurations of the most probable configuration; and

an override module to override the audio of all the other individuals in that subconfiguration of the most probable configuration while the at least one individual speaks.

4. A system according to claim 1, further comprising:

a calculation module to calculate a mean of the combined probability weights for the subconfigurations of each possible conversational configuration as the probability of that possible conversational configuration.

5. A system according to claim 1, further comprising:

an audio transmitting module to provide to at least one individual who has not yet spoken, at least one of the audio from all the determined conversations and the audio for one or more individuals in at least one of the determined conversations.

6. A system according to claim 1, further comprising:

a configuration determination module to determine at least one of the possible conversational configurations, comprising;

a representation module to represent each of the plurality of individuals as a node and to form the subconfigurations by connecting two or more nodes with at least one edge; and

a combination module to combine the subconfigurations as the possible conversational configuration.

7. A system according to claim 1, further comprising:

an individual detection module to detect audio data from a new individual;

the conversational configurations module to generate new possible conversational configurations based on the audio data from the individuals and the new individual;

the probability module to determine a new probability to each of the new possible conversational configurations; and

the selection module to select one of the new possible conversational configurations with the highest new probability.

8. A system according to claim 1, wherein the analysis of the audio data is performed using conversational characteristics shared by the plurality of individuals.

9. A computer-implemented method for enhancing audio to individuals participating in a conversation, comprising:

analyzing audio data for a plurality of individuals participating in one or more conversations;

generating based on the audio data, a plurality of possible conversational configurations of the individuals, each possible conversational configuration comprising one or more subconfigurations of at least two of the individuals;

assigning a probability weight to each of the subconfigurations comprising a likelihood that the individuals of that subconfiguration are participating in one of the conversations;

determining a probability of each possible conversational configuration by combining the probability weights for the subconfigurations of that possible conversational configuration;

selecting one of the possible conversational configurations with the highest probability as a most probable configuration;

determining the individuals participating in the conversations based on the subconfigurations of the most probable configuration; and

enhancing audio for each individual participating in the determined conversations.

10. A method according to claim 9, wherein the audio is enhanced by at least one of volume adjustment, audio effect, and spatialization control.

11. A method according to claim 9, further comprising:

providing an override privilege to at least one individual in one of the subconfigurations of the most probable configuration; and

overriding the audio of all the other individuals in that subconfiguration of the most probable configuration while the at least one individual speaks.

12. A method according to claim 9, further comprising:

calculating a mean of the combined probability weights for the subconfigurations of each possible conversational configuration as the probability of that possible conversational configuration.

13. A method according to claim 9, further comprising:

providing to at least one individual who has not yet spoken, at least one of the audio from all the determined conversations and the audio for one or more individuals in at least one of the determined conversations.

14. A method according to claim 9, further comprising:

determining at least one of the possible conversational configurations, comprising;

representing each of the plurality of individuals as a node;

forming the subconfigurations by connecting two or more nodes with at least one edge; and

combining the subconfigurations as the possible conversational configuration.

15. A method according to claim 9, further comprising:

detecting audio data from a new individual;

generating new possible conversational configurations based on the audio data from the individuals and the new individual;

determining a new probability to each of the new possible conversational configurations; and

selecting one of the new possible conversational configurations with the highest new probability.

16. A method according to claim 9, wherein the analysis of the audio data is performed using conversational characteristics shared by the plurality of individuals.

17. A non-transitory computer readable storage medium storing code for executing on a computer system to perform the method according to claim 9.

18. A computer-implemented method for enhancing visual representation to individuals participating in a conversation, comprising:

analyzing visual data for a plurality of individuals participating in one or more conversations;

generating based on the visual data, a plurality of possible conversational configurations of the individuals, each possible conversational configuration comprising one or more subconfigurations of at least two of the individuals;

assigning a probability weight to each of the subconfigurations comprising a likelihood that the individuals of that subconfiguration are participating in one of the conversations;

determining a probability of each possible conversational configuration by combining the probability weights for the subconfigurations of that possible conversational configuration;

selecting one of the possible conversational configurations with the highest probability as a most probable configuration;

determining the individuals participating in the conversations based on the subconfigurations of the most probable configuration; and

enhancing visual representation for each individual participating in the determined conversations.

19. A method according to claim 18, further comprising:

calculating a mean of the combined probability weights for the subconfigurations of each possible conversational configuration as the probability of that possible conversational configuration.

20. A method according to claim 18, further comprising:

determining at least one of the possible conversational configurations, comprising;

representing each of the plurality of individuals as a node;

forming the subconfigurations by connecting two or more nodes with at least one edge; and

combining the subconfigurations as the possible conversational configuration.