Non-linguistic signal detection and feedback

• US 8,963,987 B2
• Filed: 05/27/2010
• Issued: 02/24/2015
• Est. Priority Date: 05/27/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Byun, Byungki, Chou, Philip A., Czerwinski, Mary P., Kapoor, Ashish, Lee, Bongshin
• Primary Examiner(s): Teshale, Akelaw
• Application Number:
  US12/789,142
• Publication Number:
  US 20110292162A1
• Time in Patent Office: 1,734 Days
• Field of Search: 379 881- 8813, 379/93.21, 379/158, 379/202.01, 379/205.01, 379/102.07, 379/106.01, 348 1408- 141, 348/211.12, 455/67.7, 455/156.1, 455/416, 455/69, 370260-266
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Non-linguistic signal information relating to one or more participants to an interaction may be determined using communication data received from the one or more participants. Feedback can be provided based on the determined non-linguistic signals. The participants may be given an opportunity to opt in to having their non-linguistic signal information collected, and may be provided complete control over how their information is shared or used.

Claims

1. A method comprising:

receiving audio data of a participant to a telecommunication session, the telecommunication session including a plurality of participants in communication through a network;

using, by a processor, pattern recognition to determine at least one non-linguistic signal for the participant based on the received audio data, the at least one non-linguistic signal being representative of a behavior of the participant during the telecommunication session; and

providing information on the at least one non-linguistic signal as feedback to the participant during the telecommunication session to provide the participant an indication of the behavior and providing information on a plurality of non-linguistic signals for the participant collected across multiple interactions of the participant.

2. The method according to claim 1, wherein:

the participant is a first participant of the plurality of participants; and

a second participant of the plurality of participants also receives as feedback during the telecommunication session the information on the at least one non-linguistic signal of the first participant.

3. The method according to claim 1, further comprising providing to the participant the information on the at least one non-linguistic signal and information on other non-linguistic signals determined for the participant from multiple telecommunication sessions other than the telecommunication session in which the participant has taken part over time for identifying one or more patterns of behavior of the participant.

4. The method according to claim 1, further comprising:

receiving audio data for the plurality of participants;

determining at least one non-linguistic signal for individual participants of the plurality of participants based on the received audio data, the at least one non-linguistic signal being representative of a behavior of a corresponding participant during the telecommunication session; and

providing the at least one non-linguistic signal as feedback to the corresponding participant during the telecommunication session to provide an indication of his or her behavior.

5. The method according to claim 1, wherein the feedback identifies a behavior of the participant and provides information based on the identified behavior for improving communication, and the feedback includes an inquiry as to the accuracy of the behavior identified, the method further comprising:

receiving from the participant an indication as to the accuracy of the behavior identified; and

refining at least one of a statistical model or a pattern recognition component used to identify the behavior based on the indication received from the participant.

6. A system comprising:

one or more processors in communication with one or more computer-readable storage media;

a receiving component, maintained on the one or more computer-readable storage media and executed by the one or more processors, to receive communication data of one or more participants to an interaction, wherein the communication data includes audio data;

an analysis component to identify one or more non-linguistic signals of the one or more participants based at least in part on the audio data; and

a feedback component to determine feedback based on the one or more non-linguistic signals and based on a plurality of non-linguistic signals collected across multiple interactions of the one or more participants.

7. The system according to claim 6, wherein the communication data for a particular participant includes at least one of audio data from a microphone in proximity to the particular participant or video data from a video camera directed at the particular participant.

8. The system according to claim 6, the analysis component comprising at least one component for analyzing the audio data, comprising at least one of:

a speaking percentage component, a syllabic rate component, a speech spectrum component, a pitch variation component, a barge-in rate component, a grant-floor rate component, or an interruption suppression rate component.

9. The system according to claim 6, wherein the communication data further includes video data, and wherein the analysis component analyzes the video data, the analysis component comprising at least one of a headshake detection component or an eye tracking component.

10. The system according to claim 6, wherein:

the receiving component, the analysis component and a feedback component are implemented on a system computing device;

the system computing device is in communication with a plurality of user computing devices via a network; and

the receiving component receives the communication data for a particular participant from a particular user computing device used by the particular participant to communicate with other participants during a telecommunication session.

11. The system according to claim 10, wherein:

the feedback component provides feedback to the particular user computing device from which the communication data for the particular participant was received; and

the feedback comprises at least one non-linguistic signal interpreted from the communication data received from the particular user computing device.

12. The system according to claim 6, wherein the one or more non-linguistic signals are based on at least one of influence, consistency, or activity.

13. The system according to claim 6, further comprising a feedback component to provide feedback based on the one or more non-linguistic signals, wherein the feedback includes providing a visualization of at least one of influence, consistency, activity, or speaking percentage to be displayed on a user interface presented to at least one participant.

14. The system according to claim 6, further comprising a feedback component to provide feedback based on the one or more non-linguistic signals, wherein the feedback includes an estimate of a higher-level role of one or more of the participants.

15. The system according to claim 6, wherein:

the interaction includes a telecommunication session;

at least some of the one or more participants are located in a meeting room having a telecommunication system; and

communication data for the participants in the meeting room is correlated to particular participants based on at least one of facial recognition or locations of the participants relative to a plurality of microphones in the meeting room.

16. A method comprising:

transmitting communication data to a system computing device, the communication data corresponding to a participant participating in a telecommunication session, wherein the communication data includes audio data;

receiving information from the system computing device, the information based on one or more non-linguistic signals of the participant participating in the telecommunication session and based on a plurality of non-linguistic signals of the participant collected across multiple interactions, wherein a processor identifies one or more behavior patterns of the participant based on the information; and

displaying a user interface that includes feedback relating to the information.

17. The method according to claim 16, wherein the feedback further comprises information relating to at least one other non-linguistic signal corresponding to at least one other participant to the telecommunication session, the displaying further comprising displaying in the user interface the information relating to the at least one other non-linguistic signal.

18. The method according to claim 16, the displaying further comprising displaying information relating to an aggregation of non-linguistic signals for a plurality of participants to the telecommunication session to provide an indication of an overall reaction of the plurality of participants.

19. The method according to claim 16, further comprising adjusting an environment of a room in which the participant is present in response to the received feedback.

20. The system according to claim 16, wherein the audio data is associated with a microphone and the one or more behavior patterns of the participant comprises a barge-in rate.