Training punctuation models

• US 9,135,231 B1
• Filed: 01/31/2013
• Issued: 09/15/2015
• Est. Priority Date: 10/04/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Barra, Hugo B., Hamilton, Robert W.
• Primary Examiner(s): SINGH, SATWANT K
• Application Number:
  US13/755,702
• Time in Patent Office: 957 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/232 Orthographic correction, e.g. spell checking or vowelisation
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for customizing the punctuation style of a transcription. A method includes receiving an utterance from a user, obtaining an unpunctuated transcription of the utterance, identifying an instance within the unpunctuated transcription where a punctuation mark may be placed, identifying, using data associated with the user, one or more past instances that are similar to the identified instance, punctuating the unpunctuated transcription based at least on the one or more past instances, and presenting the punctuated transcription to the user.

Claims

1. A computer-implemented method comprising:

receiving audio data encoding an utterance of a user;

obtaining, by an automated speech recognizer, a transcription of the utterance of the user;

determining, by one or more processors, that one or more punctuation rules dictate that a punctuation mark may optionally be placed at a particular location within the transcription;

identifying, by the one or more processors, text that (i) was written by the user, (ii) is similar to the transcription, and (iii) includes a location that corresponds to the particular location within the transcription;

determining, by the one or more processors, whether the user placed the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription; and

determining, by the one or more processors, whether to place or not place the punctuation mark at the particular location within the transcription of the utterance, according to the one or more punctuation rules, based on determining whether the user placed or did not place the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription;

creating an electronic communication using the punctuated transcription, wherein the electronic communication is a type of electronic communication selected from the group consisting of an electronic mail message, a SMS text message, a social media message or post, or a word processing document; and

transmitting, by a client computing device, the electronic communication to a communication recipient.

2. The computer-implemented method of claim 1, further comprising:

developing custom punctuation rules based on the determining whether the user placed the punctuation mark at the corresponding location within the text.

3. The computer-implemented method of claim 1, wherein the text comprises one or more electronic mail messages, SMS text messages, social media messages, or word processing documents.

4. The computer-implemented method of claim 1, further comprising:

punctuating the transcription based at least on determining whether to place the punctuation mark at the particular location within the transcription; and

presenting the punctuated transcription to the user.

5. The computer-implemented method of claim 4, further comprising:

punctuating the transcription based at least on a standard punctuation rule set.

6. The computer-implemented method of claim 1, wherein the text is restricted to being exclusively a same type of electronic communication as the type of electronic communication created.

7. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving audio data encoding an utterance of a user;

obtaining, by an automated speech recognizer, a transcription of the utterance of the user;

determining, by one or more processors, that one or more punctuation rules dictate that a punctuation mark may optionally be placed at a particular location within the transcription;

identifying, by the one or more processors, text that (i) was written by the user, (ii) is similar to the transcription, and (iii) includes a location that corresponds to the particular location within the transcription;

determining, by the one or more processors, whether the user placed the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription;

determining, by the one or more processors, whether to place or not place the punctuation mark at the particular location within the transcription of the utterance, according to the one or more punctuation rules, based on determining whether the user placed or did not place the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription;

creating an electronic communication using the punctuated transcription, wherein the electronic communication is a type of electronic communication selected from the group consisting of an electronic mail message, a SMS text message, a social media message or post, or a word processing document; and

transmitting, by a client computing device, the electronic communication to a communication recipient.

8. The medium of claim 7, further comprising:

developing custom punctuation rules based on the determining whether the user placed the punctuation mark at the corresponding location within the text.

9. The medium of claim 7, wherein the text comprises one or more electronic mail messages, SMS text messages, social media messages, or word processing documents.

10. The medium of claim 7, further comprising:

punctuating the transcription based at least on determining whether to place the punctuation mark at the particular location within the transcription; and

presenting the punctuated transcription to the user.

11. The medium of claim 7, wherein the text is restricted to being exclusively a same type of electronic communication as the type of electronic communication created.

12. A system comprising:

one or more computers and one or more non-transitory storage devices storing instructions that are operable, when executed by one or more computers, to cause the one or more computers to perform operations comprising;

receiving audio data encoding an utterance of a user;

obtaining, by an automated speech recognizer, a transcription of the utterance of the user;

determining, by one or more processors, that one or more punctuation rules dictate that a punctuation mark may optionally be placed at a particular location within the transcription;

identifying, by the one or more processors, text that (i) was written by the user, (ii) is similar to the transcription, and (iii) includes a location that corresponds to the particular location within the transcription;

determining, by the one or more processors, whether the user placed the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription; and

determining, by the one or more processors, whether to place or not place the punctuation mark at the particular location within the transcription of the utterance, according to the one or more punctuation rules, based on determining whether the user placed or did not place the punctuation mark at the corresponding location within the text that was written by the user, and that is similar to the transcription;

creating an electronic communication using the punctuated transcription, wherein the electronic communication is a type of electronic communication selected from the group consisting of an electronic mail message, a SMS text message, a social media message or post, or a word processing document; and

transmitting, by a client computing device, the electronic communication to a communication recipient.

13. The system of claim 12, further comprising:

developing custom punctuation rules based on the determining whether the user placed the punctuation mark at the corresponding location within the text.

14. The system of claim 12, wherein the text comprises one or more electronic mail messages, SMS text messages, social media messages, or word processing documents.

15. The system of claim 12, further comprising:

punctuating the transcription based at least on determining whether to place the punctuation mark at the particular location within the transcription; and

presenting the punctuated transcription to the user.

16. The system of claim 15, further comprising:

punctuating the transcription based at least on a standard punctuation rule set.

17. The system of claim 12, wherein the text is restricted to being exclusively a same type of electronic communication as the type of electronic communication created.