Generating structured text content using speech recognition models

• US 10,860,685 B2
• Filed: 11/28/2016
• Issued: 12/08/2020
• Est. Priority Date: 11/28/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Co, Christopher S., Jaitly, Navdeep, Peng, Lily Hao Yi, Chou, Katherine Irene, Sankar, Ananth
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/362,643
• Publication Number:
  US 20180150605A1
• Time in Patent Office: 1,471 Days
• Field of Search: 704 9, 704270, 7042701, 704275, 704231, 704246, 704257, 704235
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media for speech recognition. One method includes obtaining an input acoustic sequence, the input acoustic sequence representing one or more utterances; processing the input acoustic sequence using a speech recognition model to generate a transcription of the input acoustic sequence, wherein the speech recognition model comprises a domain-specific language model; and providing the generated transcription of the input acoustic sequence as input to a domain-specific predictive model to generate structured text content that is derived from the transcription of the input acoustic sequence.

Claims

1. A computer implemented method comprising:

obtaining an input acoustic sequence, the input acoustic sequence representing one or more utterances;

processing the input acoustic sequence using a speech recognition model to generate a transcription of the input acoustic sequence; and

providing the generated transcription of the input acoustic sequence as input to a domain-specific predictive model to generate structured text content, wherein the domain-specific predictive model comprises a summarization predictive model configured to generate a physician note that is derived from the transcription of the input acoustic sequence and further data relating to the input acoustic sequence.

2. The method of claim 1, wherein the speech recognition model comprises a domain-specific language model.

3. The method of claim 2, wherein the domain-specific language model comprises a medical language model that has been trained using medical-specific training data.

4. The method of claim 3, wherein the domain-specific predictive model comprises an automated billing predictive model that is configured to generate a bill based on the transcription of the input acoustic sequence.

5. The method of claim 4, wherein the automated billing predictive model is further configured to generate a bill that is based on the transcription of the input acoustic sequence and one or more of (i) the input acoustic sequence, (ii) data associated with the input acoustic sequence, (iii) an acoustic sequence representing a physician dictation, or (iv) data representing a patient'"'"'s medical record.

6. The method of claim 3, wherein the domain-specific predictive model comprises a patient instruction predictive model that is configured to generate patient instructions that are derived from the transcription of the input acoustic sequence.

7. The method of claim 6, wherein the patient instruction predictive model is further configured to generate patient instructions that are derived from the transcription of the input acoustic sequence and one or more of (i) the input acoustic sequence, (ii) data associated with the input acoustic sequence, (iii) an acoustic sequence representing a physician dictation, or (iv) data representing a patient'"'"'s medical record.

8. The method of claim 3, further comprising providing the input acoustic sequence as input to a speech prosody detection predictive model configured to process the input acoustic sequence to generate an indication of speech prosody that is derived from the input acoustic sequence.

9. The method of claim 8, further comprising screening for diseases based on the generated indication of speech prosody.

10. The method of claim 3, wherein the domain-specific predictive model comprises a translation model that is configured to translate the transcription of the input acoustic sequence into a target language.

11. The method of claim 10, wherein the translation model is further configured to translate the transcription of the input acoustic sequence into a target language using the input acoustic sequence.

12. The method of claim 1, wherein the further data relating to the input acoustic sequence comprises one or more of (i) the input acoustic sequence, (ii) data associated with the input acoustic sequence, (iii) an acoustic sequence representing a physician dictation, or (iv) data representing a patient'"'"'s medical record.

13. The method of claim 12, wherein the physician note comprises a document with a particular document format.

14. The method of claim 13, wherein the physician note comprises information that the summarization predictive model has extracted from the transcription of the input acoustic sequence based on the document format.

15. The method of claim 13, wherein the physician note comprises information that the summarization predictive model has extracted from the transcription of the input acoustic sequence and augmented based on the document format.

21. The method of claim 9, wherein the speech prosody detection predictive model is configured to provide, as output, a document listing results from the screening.

16. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

obtaining an input acoustic sequence, the input acoustic sequence representing one or more utterances;

processing the input acoustic sequence using a speech recognition model to generate a transcription of the input acoustic sequence; and

providing the generated transcription of the input acoustic sequence as input to a domain-specific predictive model to generate structured text content, wherein the domain-specific predictive model comprises a summarization predictive model configured to generate a physician note that is derived from the transcription of the input acoustic sequence and further data relating to the input acoustic sequence.

17. The system of claim 16, wherein the speech recognition model comprises a domain specific language model.

18. The system of claim 16, wherein the speech recognition model comprises a hybrid deep neural network—

hidden Markov model automatic speech recognition model.

19. The system of claim 16, wherein the speech recognition model comprises an end-to-end speech recognition model with attention.

20. A computer-readable storage medium comprising instructions stored thereon that are executable by a processing device and upon such execution cause the processing device to perform operations comprising:

obtaining an input acoustic sequence, the input acoustic sequence representing one or more utterances;

processing the input acoustic sequence using a speech recognition model to generate a transcription of the input acoustic sequence; and

providing the generated transcription of the input acoustic sequence as input to a domain-specific predictive model to generate structured text content, wherein the domain-specific predictive model comprises a summarization predictive model configured to generate a physician note that is derived from the transcription of the input acoustic sequence and further data relating to the input acoustic sequence.