Transcription of communications using multiple speech recognition systems

• US 10,192,554 B1
• Filed: 02/26/2018
• Issued: 01/29/2019
• Est. Priority Date: 02/26/2018
• Status: Active Grant

• Current Assignee: Sorenson Ip Holdings, LLC
• Original Assignee: Sorenson Ip Holdings, LLC
• Inventors: Boehme, Kenneth, Holm, Michael, Roylance, Shane
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/905,180
• Time in Patent Office: 337 Days
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A method may include obtaining audio data originating at a first device during a communication session between the first device and a second device and providing the audio data to a first speech recognition system to generate a first transcript based on the audio data and directing the first transcript to the second device. The method may also include in response to obtaining a quality indication regarding a quality of the first transcript, multiplexing the audio data to provide the audio data to a second speech recognition system to generate a second transcript based on the audio data while continuing to provide the audio data to the first speech recognition system and direct the first transcript to the second device, and in response to obtaining a transfer indication that occurs after multiplexing of the audio data, directing the second transcript to the second device instead of the first transcript.

Claims

1. A method to transcribe communications, the method comprising:

obtaining audio data originating at a first device during a communication session between the first device and a second device, the communication session configured for verbal communication;

providing the audio data to a first automated speech recognition system that works independent of human interaction to generate a first transcript using the audio data;

directing the first transcript to the second device;

in response to obtaining a quality indication that indicates a quality of the first transcript is below a quality threshold and while continuing to provide the audio data to the first automated speech recognition system to generate the first transcript and continuing to direct the first transcript to the second device, the method including;

multiplexing the audio data to provide the audio data to the first automated speech recognition system and a second automated speech recognition system;

broadcasting, by the second automated speech recognition system, audio based on the multiplexed audio data;

obtaining, by the second automated speech recognition system, second audio data based on a re-voicing of the broadcast audio; and

generating, by the second automated speech recognition system, a second transcript using the second audio data; and

in response to a transfer indication that occurs after multiplexing of the audio data, directing the second transcript to the second device instead of directing the first transcript to the second device and ceasing providing the audio data to the first automated speech recognition system.

2. The method of claim 1, further comprising:

obtaining a confidence score of the first transcript from the first automated speech recognition system; and

obtaining the quality indication based on a comparison of the confidence score to the quality threshold.

3. The method of claim 1, wherein the quality indication is obtained from the second device.

4. The method of claim 1, wherein the transfer indication includes an occurrence of one of the following:

after multiplexing of the audio data, for a time period the audio data does not include spoken words that result in text in the first transcript,

the second transcript including text,

after multiplexing of the audio data, the first transcript includes first text and then for a second time period the audio data does not include spoken words that result in text in the first transcript, and

a last phrase of the first transcript is the same as a last phrase of the second transcript.

5. At least one non-transitory computer-readable media configured to store one or more instructions that when executed by at least one processor cause or direct a system to perform the method of claim 1.

6. A method to transcribe communications, the method comprising:

obtaining audio data originating at a first device during a communication session between the first device and a second device, the communication session configured for verbal communication;

providing the audio data to a first speech recognition system to generate a first transcript based on the audio data;

directing the first transcript to the second device;

in response to obtaining a quality indication that indicates a quality of the first transcript is below a quality threshold, multiplexing the audio data to provide the audio data to a second speech recognition system to generate a second transcript based on the audio data while continuing to provide the audio data to the first speech recognition system to generate the first transcript and continuing to direct the first transcript to the second device; and

in response to a transfer indication that occurs after multiplexing of the audio data and that results from an occurrence of an event that indicates the second transcript is to be directed to the second device instead of the first transcript, directing the second transcript to the second device instead of the first transcript.

7. The method of claim 6, wherein the first speech recognition system and the second speech recognition system are automated speech recognition systems that work independent of human interaction.

8. The method of claim 6, wherein the first speech recognition system is an automated speech recognition system that works independent of human interaction and the generation of the second transcript by the second speech recognition system includes:

broadcasting audio based on the audio data; and

obtaining second audio data based on a re-voicing of the broadcast audio, wherein the second transcript is generated based on the second audio data.

9. The method of claim 6, further comprising:

obtaining a confidence score of the first transcript from the first speech recognition system; and

obtaining the quality indication based on a comparison of the confidence score to the quality threshold.

10. The method of claim 6, wherein the quality indication is obtained from the second device.

11. The method of claim 6, wherein the transfer indication includes an occurrence of one of the following:

after multiplexing of the audio data, for a time period the audio data does not include spoken words that result in text in the first transcript,

the second transcript including text,

after multiplexing of the audio data, the first transcript includes first text and then for a second time period the audio data does not include spoken words that result in text in the first transcript, and

a last phrase of the first transcript is the same as a last phrase of the second transcript.

12. The method of claim 6, further comprising in response to obtaining a transfer indication that occurs after multiplexing of the audio data, ceasing providing the audio data to the first speech recognition system.

13. At least one non-transitory computer-readable media configured to store one or more instructions that when executed by at least one processor cause or direct a system to perform the method of claim 6.

14. A system comprising:

at least one processor; and

at least one non-transitory computer-readable media communicatively coupled to the at least one processor and configured to store one or more instructions that when executed by the at least one processor cause or direct the system to perform operations comprising;

obtain audio data originating at a first device during a communication session between the first device and a second device, the communication session configured for verbal communication;

provide the audio data to a first speech recognition system to generate a first transcript based on the audio data;

direct the first transcript to the second device;

in response to obtaining a quality indication that indicates a quality of the first transcript is below a quality threshold, multiplex the audio data to provide the audio data to a second speech recognition system to generate a second transcript based on the audio data while continuing to provide the audio data to the first speech recognition system to generate the first transcript and continuing to direct the first transcript to the second device; and

in response to a transfer indication that occurs after multiplexing of the audio data and that results from an occurrence of an event that indicates the second transcript is to be directed to the second device instead of the first transcript, direct the second transcript to the second device instead of the first transcript.

15. The system of claim 14, wherein the first speech recognition system and the second speech recognition system are automated speech recognition systems that work independent of human interaction.

16. The system of claim 14, wherein the first speech recognition system is an automated speech recognition system that works independent of human interaction and the generation of the second transcript by the second speech recognition system includes operations comprising:

broadcast audio based on the audio data; and

obtain second audio data based on a re-voicing of the broadcast audio, wherein the second transcript is generated based on the second audio data.

17. The system of claim 14, wherein the operations further comprise:

obtaining a confidence score of the first transcript from the first speech recognition system; and

obtaining the quality indication based on a comparison of the confidence score to the quality threshold.

18. The system of claim 14, wherein the quality indication is obtained from the second device.

19. The system of claim 14, wherein the transfer indication includes an occurrence of one of the following:

after multiplexing of the audio data, for a time period the audio data does not include spoken words that result in text in the first transcript,

the second transcript including text,

after multiplexing of the audio data, the first transcript includes first text and then for a second time period the audio data does not include spoken words that result in text in the first transcript, and

a last phrase of the first transcript is the same as a last phrase of the second transcript.

20. The system of claim 14, wherein the operations further comprise in response to obtaining a transfer indication that occurs after multiplexing of the audio data, cease providing the audio data to the first speech recognition system.