IDENTIFYING CORRESPONDING POSITIONS IN DIFFERENT REPRESENTATIONS OF A TEXTUAL WORK

• US 20140278429A1
• Filed: 03/13/2013
• Published: 09/18/2014
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ganong, William F. III
• Granted Patent: US 9,378,739 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

Described herein are techniques for determining corresponding positions between different representations of a textual work. In some of the techniques, portions of one or more representations may be processed. A determination of a corresponding position may be made in response to a request received from a user, such as a reader that desires to switch between representations. The request may indicate a position in one representation and the representation to which the user would like to switch. In response to receiving the request, one or more portions of one or more representations of a textual work may be processed. In some techniques, a corresponding position between different representations may be determined without processing the entirety of one or more representations of the textual work. For example, a corresponding position may be determined without processing an entire audio representation.

Claims

1. A method for use in identifying corresponding positions in an audio representation of a textual work and an electronic representation of the textual work, the method comprising:

performing automatic speech recognition (ASR) on a first audio segment appearing in the audio representation at a first audio position to generate a first textual representation of the first audio segment; and

identifying a first text position corresponding to the first audio position by searching the electronic representation of the textual work for text matching the first textual representation.

2. The method of claim 1, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text exactly matching the first textual representation.

3. The method of claim 1, wherein the searching the electronic representation of the textual work comprises performing a fuzzy search of the textual work.

4. The method of claim 1, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text having an edit distance, when compared to the first textual representation, below a threshold.

5. The method of claim 1, further comprising:

determining whether a mapping of corresponding positions between the audio representation and the electronic representation includes an entry that includes a position matching the first audio position or the first text position,

wherein the performing ASR on the first audio segment is performed in response to determining that the mapping does not include an entry that includes a position matching the first audio position or matching the first text position.

6. The method of claim 1, further comprising:

performing ASR, using an ASR engine, on a second audio segment appearing in the audio representation at a second audio position to generate a second textual representation of the second audio segment, wherein the performing the ASR on the second audio segment comprises generating a confidence value indicating a confidence of the ASR engine that the second textual representation is an accurate representation of the second audio segment; and

in response to determining that the confidence value is below a threshold, selecting the first audio segment appearing in the audio representation at the first audio position,

wherein the performing ASR on the first audio segment is performed in response to selecting the first audio segment.

7. The method of claim 6, wherein:

the first audio segment includes audio appearing at a position in the audio segment; and

the selecting the first audio segment comprises selecting an audio segment that includes the audio appearing at the position, the first audio segment being different from the second audio segment.

8. The method of claim 1, further comprising, prior to the performing ASR and the identifying:

identifying at least one word and/or at least one phrase included in the electronic representation of the textual work that is not included in a set of words and/or phrases used by an ASR engine that performs the ASR; and

adding the at least one word and/or the at least one phrase to the set for use by the ASR engine in performing the ASR on one or more segments of the audio representation.

9. The method of claim 1, wherein the performing ASR on the first audio segment is performed in response to receiving a request for the first text position corresponding to the first audio position, the request identifying the first audio position.

10. The method of claim 1, wherein the performing ASR on the first audio segment is performed in response to receiving a request for the first audio position corresponding to a text position, the request identifying the text position.

11. At least one computer-readable storage medium having encoded thereon computer-executable instructions that, when executed by a computer, cause the computer to carry out a method for use in identifying corresponding positions in an audio representation of a textual work and an electronic representation of the textual work, the method comprising:

performing automatic speech recognition (ASR) on a first audio segment appearing in the audio representation at a first audio position to generate a first textual representation of the first audio segment; and

identifying a first text position corresponding to the first audio position by searching the electronic representation of the textual work for text matching the first textual representation.

12. The at least one computer-readable storage medium of claim 11, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text exactly matching the first textual representation.

13. The at least one computer-readable storage medium of claim 11, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text having an edit distance, when compared to the first textual representation, below a threshold.

14. The at least one computer-readable storage medium of claim 11, wherein the method further comprises:

performing ASR, using an ASR engine, on a second audio segment appearing in the audio representation at a second audio position to generate a second textual representation of the second audio segment, wherein the performing the ASR on the second audio segment comprises generating a confidence value indicating a confidence of the ASR engine that the second textual representation is an accurate representation of the second audio segment; and

in response to determining that the confidence value is below a threshold, selecting the first audio segment appearing in the audio representation at the first audio position,

wherein the performing ASR on the first audio segment is performed in response to selecting the first audio segment.

15. The at least one computer-readable storage medium of claim 11, wherein the method further comprises, prior to the performing ASR and the identifying:

identifying at least one word and/or at least one phrase included in the electronic representation of the textual work that is not included in a set of words and/or phrases used by an ASR engine that performs the ASR; and

adding the at least one word and/or the at least one phrase to the set for use by the ASR engine in performing the ASR on one or more segments of the audio representation.

16. An apparatus comprising:

at least one processor; and

at least one computer-readable storage medium having encoded thereon computer-executable instructions that, when executed by a computer, cause the computer to carry out a method for use in identifying corresponding positions in an audio representation of a textual work and an electronic representation of the textual work, the method comprising;

performing automatic speech recognition (ASR) on a first audio segment appearing in the audio representation at a first audio position to generate a first textual representation of the first audio segment; and

identifying a first text position corresponding to the first audio position by searching the electronic representation of the textual work for text matching the first textual representation.

17. The apparatus of claim 16, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text exactly matching the first textual representation.

18. The apparatus of claim 16, wherein the searching the electronic representation of the textual work for text matching the first textual representation comprises searching the electronic representation of the textual work for text having an edit distance, when compared to the first textual representation, below a threshold.

19. The apparatus of claim 16, wherein the method further comprises:

performing ASR, using an ASR engine, on a second audio segment appearing in the audio representation at a second audio position to generate a second textual representation of the second audio segment, wherein the performing the ASR on the second audio segment comprises generating a confidence value indicating a confidence of the ASR engine that the second textual representation is an accurate representation of the second audio segment; and

in response to determining that the confidence value is below a threshold, selecting the first audio segment appearing in the audio representation at the first audio position,

wherein the performing ASR on the first audio segment is performed in response to selecting the first audio segment.

20. The apparatus of claim 16, wherein the method further comprises, prior to the performing ASR and the identifying:

identifying at least one word and/or at least one phrase included in the electronic representation of the textual work that is not included in a set of words and/or phrases used by an ASR engine that performs the ASR; and

adding the at least one word and/or the at least one phrase to the set for use by the ASR engine in performing the ASR on one or more segments of the audio representation.