Computer-implemented system and method for transcribing verbal messages

• US 9,418,659 B2
• Filed: 11/07/2013
• Issued: 08/16/2016
• Est. Priority Date: 03/28/2002
• Status: Expired due to Term

• Current Assignee: Avaya LLC (Avaya Incorporated), Avaya Management L.P. (Avaya Incorporated)
• Original Assignee: Intellisist, Inc. dba Spoken Communications (Avaya Incorporated)
• Inventors: Webb, Mike O.
• Primary Examiner(s): Shah, Paras D
• Application Number:
  US14/074,653
• Publication Number:
  US 20140067390A1
• Time in Patent Office: 1,013 Days
• Field of Search: 704/231, 704/235, 704/239, 704/251, 704/254, 704/270, 704/275, 379 8801- 8828, 700 99-111
• US Class Current: 1/1
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2203/2016 Call initiation by network rather than by subscriber
• CPC Class Codes: H04M 2203/4536 Voicemail combined with text-based messaging
• CPC Class Codes: H04M 3/53333 Message receiving aspects

Abstract

A computer-implemented system and method for transcribing verbal messages is provided. Verbal messages each comprising audio content are received. Automatically recognized text is generated for the audio content of at least one of the verbal messages. A turn-around processing time is applied to the verbal message. The automatically recognized text and verbal message are transferred to a human agent when an expected processing time of the verbal message satisfies the turn-around processing time. At least a portion of the automatically recognized text is replaced with manual transcription from the human agent. The automatically recognized text and manual transcription are provided as a text message.

Claims

1. A computer-implemented system for transcribing verbal messages, comprising:

an inbound message processor to receive verbal messages each comprising audio content;

a split and merge message processor to divide the verbal message into segments prior to generating automatically recognized text;

a speech recognizer to generate the automatically recognized text for the audio content of at least one of the verbal messages and to obtain the automatically recognized text for each segment;

a pattern matcher to determine whether two or more of the segments share a common format;

a transmission module to provide the segments without a common format to a human agent and at least one other human agent;

a time threshold module to apply a turn-around processing time to the verbal message;

a transfer module to transfer the automatically recognized text and verbal message to the human agent when an expected processing time of the verbal message is below the turn-around processing time;

an editing module to replace at least a portion of the automatically recognized text with manual transcription from the human agent;

a post-processor to generate post-processed text from the automatically recognized text of the segments that share a common format, of the automatically recognized text of the segments without a common format that have not been replaced, and of the manual transcription from the human agent; and

a transmission module to perform at least one of transmitting the post-processed text to a user via email, delivering the post-processed text to a user via short message service, and supplying the post-processed text to a user via an application program interface.

2. A system according to claim 1, further comprising:

an agent selection module to select the human agent for sending the automatically recognized text and the audio content for the verbal message based on at least one of an agent rank, availability, and message content.

3. A system according to claim 2, wherein the rank comprises at least one of agent fatigue, performance, and speed.

4. A system according to claim 1, further comprising:

a message transmission module to simultaneously send the verbal message to the speech recognition module for automatic speech recognition and to the human agent for manual transcription.

5. A system according to claim 1, wherein the manual transcription is performed by the human agent using at least one of a line mode that allows the human agent to focus on a portion of the verbal message, a word mode that includes shortcuts on a keyboard for editing or transcribing the verbal message, and a whole message mode that enables the human agent to edit or transcribe the entire verbal message.

6. A system according to claim 1, further comprising:

a segment module to divide the verbal message into segments prior to generating the automatically recognized text;

the speech recognition module to obtain the automatically recognized text for each segment;

a confidence rating module to assign a confidence level to the automatically recognized text for each segment;

a segment transmission module to transmit those segments with confidence levels below a confidence threshold to the human agent and at least one other human agent; and

a text entry module to enter those segments of automatically recognized text with confidence levels that satisfy the confidence threshold into the text message.

7. A system according to claim 6, further comprising:

a priority transmission module to first, send the segments with the lowest confidence levels to the human agent and later, to send the segments with higher confidence levels that do not satisfy the confidence threshold to the human agent.

8. A system according to claim 1, further comprising:

a content determination module to determine a difficulty of processing each segment;

a transmission module to assign those segments with the highest difficulty to the human agent and to subsequently assign those segments with lower difficulty to the human agent.

9. A computer-implemented method for transcribing verbal messages, comprising:

receiving verbal messages each comprising audio content;

generating automatically recognized text for the audio content of at least one of the verbal messages;

dividing the verbal message into segments prior to generating the automatically recognized text;

obtaining the automatically recognized text for each segment;

determining whether two or more of the segments share a common format;

providing the segments without a common format to a human agent and at least one other human agent; and

applying a turn-around processing time to the verbal message;

transferring the automatically recognized text and the verbal message to the human agent when an expected processing time of the verbal message is below the turn-around processing time;

replacing at least a portion of the automatically recognized text with manual transcription from the human agent;

generating, via a post-processor, post-processed text from the automatically recognized text of the segments that share a common format, of the automatically recognized text of the segments without a common format that have not been replaced, and of the manual transcription from the human agent; and

performing at least one of transmitting the post-processed text to a user via email, delivering the post-processed text to a user via short message service, and supplying the post-processed text to a user via an application program interface.

10. A method according to claim 9, further comprising:

selecting the human agent for sending the automatically recognized text and the audio content for the verbal message based on at least one of an agent rank, availability, and message content.

11. A method according to claim 10, wherein the rank comprises at least one of agent fatigue, performance, and speed.

12. A method according to claim 9, further comprising:

simultaneously sending the verbal message to a speech recognition module for automatic speech recognition and to the human agent for manual transcription.

13. A method according to claim 9, wherein the manual transcription is performed by the human agent using at least one of a line mode that allows the human agent to focus on a portion of the verbal message, a word mode that includes shortcuts on a keyboard for editing or transcribing the verbal message, and a whole message mode that enables the human agent to edit or transcribe the entire verbal message.

14. A method according to claim 9, further comprising:

dividing the verbal message into segments prior to generating the automatically recognized text;

obtaining the automatically recognized text for each segment;

assigning a confidence rating to the automatically recognized text for each segment;

transmitting those segments with confidence levels below a confidence threshold to the human agent and at least one other human agent; and

entering those segments of automatically recognized text with confidence levels that satisfy the confidence threshold into the text message.

15. A method according to claim 14, further comprising:

first, sending the segments with the lowest confidence levels to the human agent; and

later, sending the segments with higher confidence levels that do not satisfy the confidence threshold to the human agent.

16. A method according to claim 9, further comprising:

determining a difficulty of processing each segment;

assigning those segments with the highest difficulty to the human agent; and

subsequently assigning those segments with lower difficulty to the human agent.