Training speech recognition systems using word sequences

• US 10,388,272 B1
• Filed: 12/04/2018
• Issued: 08/20/2019
• Est. Priority Date: 12/04/2018
• Status: Active Grant

• Current Assignee: Sorenson Ip Holdings, LLC
• Original Assignee: Sorenson Ip Holdings, LLC
• Inventors: Thomson, David, Adams, Jadie, Boehme, Kenneth
• Primary Examiner(s): Singh, Satwant K
• Application Number:
  US16/209,640
• Time in Patent Office: 259 Days
• Field of Search: 704235
• CPC Class Codes: G06F 21/6254 by anonymising data, e.g. decorrelating personal data from the owner's identification
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A method may include obtaining first audio data of a communication session between a first device and a second device, obtaining a text string that is a transcription of the first audio data, and selecting a contiguous sequence of words from the text string as a first word sequence. The method may further include comparing the first word sequence to multiple word sequences obtained before the communication session and in response to the first word sequence corresponding to one of the multiple word sequences, incrementing a counter of multiple counters associated with the one of the multiple word sequences. The method may also include deleting the text string and the first word sequence and training and after deleting the text string and the first word sequence, training a language model of an automatic transcription system using the multiple word sequences and the multiple counters.

Claims

1. A method comprising:

obtaining first audio data of a communication session between a first device of a first user and a second device of a second user, the communication session configured for verbal communication;

obtaining, during the communication session, a text string that is a transcription of the first audio data from an automatic transcription system;

selecting, during the communication session, a contiguous sequence of words from the text string as a first word sequence;

comparing, during the communication session, the first word sequence to a plurality of word sequences obtained before the communication session, each of the plurality of word sequences associated with a corresponding one of a plurality of counters;

in response to the first word sequence corresponding to one of the plurality of word sequences based on the comparison, incrementing, during the communication session, a counter of the plurality of counters associated with the one of the plurality of word sequences;

after incrementing the counter of the plurality of counters, deleting the text string and the first word sequence, wherein the first word sequence is deleted during the communication session; and

training, after deleting the text string and the first word sequence, a language model of the automatic transcription system using the plurality of word sequences and the plurality of counters.

2. The method of claim 1, wherein each one of the plurality of counters indicates a number of occurrences that a corresponding one of the plurality of words sequences is included in a plurality of transcriptions of a plurality of communication sessions that occur between a plurality of devices, the plurality of devices not including the first device and the second device.

3. The method of claim 1, wherein the text string is deleted during the communication session.

4. The method of claim 1, wherein the text string is denormalized before selecting the contiguous sequence of words as the first word sequence.

5. The method of claim 1, further comprising:

selecting a second contiguous sequence of words from the text string as a second word sequence;

comparing the second word sequence to the plurality of word sequences; and

in response to the second word sequence not corresponding to any of the plurality of word sequences based on the comparison, adding a third word sequence based on the second word sequence to the plurality of word sequences and adding a second counter with a count of one to the plurality of counters that is associated with the third word sequence of the plurality of word sequences,

wherein the training the language model of the automatic transcription system using the plurality of word sequences and the plurality of counters occurs after adding the second word sequence to the plurality of word sequences.

6. The method of claim 5, wherein

the third word sequence is the same as the second word sequence,

the third word sequence includes fewer words than the second word sequence, or

the third word sequence includes a replacement word that is a generic word of one of the words in the second word sequence, the replacement word used in place of the one of the words in the second word sequence such that the third word sequence and the second word sequence include a same number of words.

7. The method of claim 6, wherein the one of the words in the second word sequence are replaced based on the one of the words meeting a sensitive criteria, and

wherein removal words removed from the second word sequence to generate the third word sequence are removed based on the removal words meeting the sensitive criteria, wherein the third word sequence includes fewer words than the second word sequence.

8. The method of claim 6, further comprising:

adding the one of the words in the second word sequence that is replaced by the replacement word to the plurality of word sequences; and

adding a third counter with a count of one to the plurality of counters that is associated with the one of the words in the second word sequence.

9. At least one non-transitory computer readable media configured to store one or more instructions that in response to be executed by at least one computing system cause performance of the method of claim 1.

10. A method comprising:

obtaining first audio data of a communication session between a first device of a first user and a second device of a second user, the communication session configured for verbal communication;

obtaining, during the communication session, a text string that is a transcription of the first audio data;

selecting a contiguous sequence of words from the text string as a first word sequence;

comparing the first word sequence to a plurality of word sequences obtained before the communication session, each of the plurality of word sequences associated with a corresponding one of a plurality of counters;

in response to the first word sequence corresponding to one of the plurality of word sequences based on the comparison, incrementing a counter of the plurality of counters associated with the one of the plurality of word sequences;

after incrementing the counter of the plurality of counters, deleting the text string and the first word sequence; and

training, after deleting the text string and the first word sequence, a language model of an automatic transcription system using the plurality of word sequences and the plurality of counters.

11. The method of claim 10, wherein the text string is obtained from the automatic transcription system.

12. The method of claim 10, wherein each one of the plurality of counters indicates a number of occurrences that a corresponding one of the plurality of words sequences is included in a plurality of transcriptions of a plurality of communication sessions that occur between a plurality of devices, the plurality of devices not including the first device and the second device.

13. The method of claim 10, wherein the steps of selecting the contiguous sequence of words, comparing the first word sequence, and incrementing the counter of the plurality of counters, each occur during the communication session.

14. The method of claim 10, wherein the text string and the first word sequence are deleted during the communication session.

15. The method of claim 10, wherein the text string is denormalized before selecting the contiguous sequence of words as the first word sequence.

16. The method of claim 10, further comprising:

selecting a second contiguous sequence of words from the text string as a second word sequence;

comparing the second word sequence to the plurality of word sequences; and

in response to the second word sequence not corresponding to any of the plurality of word sequences based on the comparison, adding a third word sequence based on the second word sequence to the plurality of word sequences and adding a second counter with a count of one to the plurality of counters that is associated with the third word sequence of the plurality of word sequences,

wherein the training the language model of the automatic transcription system using the plurality of word sequences and the plurality of counters occurs after adding the second word sequence to the plurality of word sequences.

17. The method of claim 16, wherein

the third word sequence is the same as the second word sequence,

the third word sequence includes fewer words than the second word sequence, or

the third word sequence includes a replacement word that is a generic word of one of the words in the second word sequence, the replacement word used in place of the one of the words in the second word sequence such that the third word sequence and the second word sequence include a same number of words.

18. The method of claim 17, wherein the one of the words in the second word sequence are replaced based on the one of the words meeting a sensitive criteria, and

wherein removal words removed from the second word sequence to generate the third word sequence that includes fewer words than the second word sequence are removed based on the removal words meeting the sensitive criteria.

19. The method of claim 17, further comprising:

adding the one of the words in the second word sequence that is replaced by the replacement word to the plurality of word sequences; and

adding a third counter with a count of one to the plurality of counters that is associated with the one of the words in the second word sequence.

20. At least one non-transitory computer readable media configured to store one or more instructions that in response to be executed by at least one computing system cause performance of the method of claim 10.