Multimodal Teleconferencing

• US 20110032845A1
• Filed: 08/05/2009
• Published: 02/10/2011
• Est. Priority Date: 08/05/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Cross, Charles W. JR., Agapi, Ciprian, Bodin, William K.
• Granted Patent: US 8,416,714 B2
• Time in Patent Office: Days
• US Class Current: 370/260
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: H04L 12/413 with random access, e.g. carrier-sense multiple-access with collision detection [CSMA-CD]

Abstract

Multimodal teleconferencing including receiving, by a multimodal teleconferencing module, a speech utterance from one of a plurality of participants in the multimodal teleconference; identifying the participant making the speech utterance as a current speaker; retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker; retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference; providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the current speaker, an identification of the speaker and the content retrieved for the speaker; and providing, by the multimodal teleconferencing module to one or more of multimodal teleconferencing clients for display to the other participants, an identification of the current speaker with the content retrieved for the one or more other participants in the multimodal teleconference.

Claims

1. A method of multimodal teleconferencing, the method implemented with the multimodal application and a multimodal teleconferencing module, a module of automated computing machinery operating on a server supporting a voice mode including accepting speech input from participants of the multimodal teleconference through a telephony interface, digitizing the speech, and providing digitized speech to a speech engine, the method comprising:

receiving, by the multimodal teleconferencing module, a speech utterance from one of a plurality of participants in the multimodal teleconference;

identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference;

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the current speaker, an identification of the speaker and the content retrieved for the speaker; and

providing, by the multimodal teleconferencing module to one or more of multimodal teleconferencing clients for display to the other participants, an identification of the current speaker with the content retrieved for the one or more other participants in the multimodal teleconference.

2. The method of claim 1 further comprising:

receiving, by the multimodal teleconferencing module, speech from the current speaker;

creating, by the multimodal teleconferencing module, text from the speech; and

providing, by the multimodal teleconferencing module to the multimodal teleconferencing clients for all participants in the multimodal teleconference, the text as a transcript of the multimodal teleconference.

3. The method of claim 1 wherein identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker further comprises:

receiving speech from the current speaker; and

determining whether the speech matches the voiceprint of a participant of the multimodal teleconference.

4. The method of claim 1 wherein identifying the caller as a valid participant in the multimodal telephone conference further comprises:

receiving speech from the current speaker;

identifying the telephone number of the current speaker; and

determining whether the phone number of the current speaker matches a known number of a participant of the multimodal teleconference.

5. The method of claim 1 further comprising:

receiving, by the multimodal teleconferencing client of listening participant other than the current speaker, a speech utterance identifying another particular participant of interest;

requesting, by the multimodal teleconferencing client from the multimodal teleconferencing module;

content of the another particular participant of interest;

retrieving, by the multimodal teleconferencing module from accounts for the another particular participant of interest, content of the another particular participant of interest;

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the listening participant, the content of the another particular participant of interest; and

displaying, by the multimodal teleconferencing client to the listening participant, the content of the another particular participant of interest.

6. The method of claim 1 further comprising:

receiving, by the multimodal teleconferencing client of listening participant other than the current speaker through a non-voice mode user input device, an identification of another particular participant of interest;

requesting, by the multimodal teleconferencing client from the multimodal teleconferencing module;

content of the another particular participant of interest;

retrieving, by the multimodal teleconferencing module from accounts for the another particular participant of interest, content of the another particular participant of interest;

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the listening participant, the content of the another particular participant of interest; and

displaying, by the multimodal teleconferencing client to the listening participant, the content of the another particular participant of interest.

8. The apparatus of claim 1 wherein the computer memory also has disposed within it computer program instructions for:

receiving, by the multimodal teleconferencing module, speech from the current speaker;

creating, by the multimodal teleconferencing module, text from the speech; and

providing, by the multimodal teleconferencing module to the multimodal teleconferencing clients for all participants in the multimodal teleconference, the text as a transcript of the multimodal teleconference.

9. The apparatus of claim 1 wherein computer program instructions for identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker further comprise computer program instructions for:

receiving speech from the current speaker; and

determining whether the speech matches the voiceprint of a participant of the multimodal teleconference.

10. The apparatus of claim 1 wherein computer program instructions for identifying the caller as a valid participant in the multimodal telephone conference further comprise computer program instructions for:

receiving speech from the current speaker;

identifying the telephone number of the current speaker; and

determining whether the phone number of the current speaker matches a known number of a participant of the multimodal teleconference.

7. A apparatus of multimodal teleconferencing, the apparatus including the multimodal application and a multimodal teleconferencing module, a module of automated computing machinery operating on a server supporting a voice mode including accepting speech input from participants of the multimodal teleconference through a telephony interface, digitizing the speech, and providing digitized speech to a speech engine, the apparatus comprising a computer memory operatively coupled to a computer processor, the memory having disposed within it computer program instructions for:

receiving, by the multimodal teleconferencing module, a speech utterance from one of a plurality of participants in the multimodal teleconference;

identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference;

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the current speaker, an identification of the speaker and the content retrieved for the speaker; and

providing, by the multimodal teleconferencing module to one or more of multimodal teleconferencing clients for display to the other participants, an identification of the current speaker with the content retrieved for the one or more other participants in the multimodal teleconference.

11. The apparatus of claim 7 wherein the memory having disposed within it computer program instructions for:

retrieving, by the multimodal teleconferencing module from accounts for the another particular participant of interest at the request of a multimodal teleconferencing client of listening participant other than the current speaker, content of the another particular participant of interest; and

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the listening participant, the content of the another particular participant of interest.

12. The apparatus of claim 7 wherein computer program instructions for retrieving by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker include computer program instructions for retrieving email for the current speaker concerning the multimodal teleconference, biographical information about the current speaker, and documents of the current speaker related to the multimodal teleconference.

13. The apparatus of claim 7 wherein computer program instructions for retrieving by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference include computer program instructions for retrieving email for the current speaker concerning the multimodal teleconference, biographical information about the current speaker, and documents of the current speaker related to the multimodal teleconference.

14. A computer program product of multimodal teleconferencing, the computer program product including a module of automated computing machinery operating on a server supporting a voice mode including accepting speech input from participants of the multimodal teleconference through a telephony interface, digitizing the speech, and providing digitized speech to a speech engine, the computer program product disposed on a recording medium, the computer program product comprising computer program instructions for:

receiving, by the multimodal teleconferencing module, a speech utterance from one of a plurality of participants in the multimodal teleconference;

identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker;

retrieving, by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference;

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the current speaker, an identification of the speaker and the content retrieved for the speaker; and

providing, by the multimodal teleconferencing module to one or more of multimodal teleconferencing clients for display to the other participants, an identification of the current speaker with the content retrieved for the one or more other participants in the multimodal teleconference.

15. The computer program product of claim 14 wherein the computer memory also has disposed within it computer program instructions for:

receiving, by the multimodal teleconferencing module, speech from the current speaker;

creating, by the multimodal teleconferencing module, text from the speech; and

providing, by the multimodal teleconferencing module to the multimodal teleconferencing clients for all participants in the multimodal teleconference, the text as a transcript of the multimodal teleconference.

16. The computer program product of claim 14 wherein computer program instructions for identifying, by the multimodal teleconferencing module, the participant making the speech utterance as a current speaker further comprise computer program instructions for:

receiving speech from the current speaker; and

determining whether the speech matches the voiceprint of a participant of the multimodal teleconference.

17. The computer program product of claim 14 wherein computer program instructions for identifying the caller as a valid participant in the multimodal telephone conference further comprise computer program instructions for:

receiving speech from the current speaker;

identifying the telephone number of the current speaker; and

determining whether the phone number of the current speaker matches a known number of a participant of the multimodal teleconference.

18. The computer program product of claim 14 wherein computer program instructions for retrieving by the multimodal teleconferencing module from accounts for the current speaker, content for display to the current speaker include computer program instructions for retrieving email for the current speaker concerning the multimodal teleconference, biographical information about the current speaker, and documents of the current speaker related to the multimodal teleconference.

19. The computer program product of claim 14 wherein computer program instructions for retrieving by the multimodal teleconferencing module from accounts for the current speaker, content for display to one or more other participants in the multimodal teleconference include computer program instructions for retrieving email for the current speaker concerning the multimodal teleconference, biographical information about the current speaker, and documents of the current speaker related to the multimodal teleconference.

20. The computer program product of claim 14 further comprising computer program instructions for:

retrieving, by the multimodal teleconferencing module from accounts for the another particular participant of interest at the request of a multimodal teleconferencing client of listening participant other than the current speaker, content of the another particular participant of interest; and

providing, by the multimodal teleconferencing module to a multimodal teleconferencing client for display to the listening participant, the content of the another particular participant of interest.