Apparatus, method, and computer program product for correcting speech recognition error

• US 10,614,265 B2
• Filed: 12/21/2016
• Issued: 04/07/2020
• Est. Priority Date: 03/16/2016
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Fume, Kosei, Ashikawa, Taira, Ashikawa, Masayuki
• Primary Examiner(s): Mills, Frank D
• Application Number:
  US15/387,367
• Publication Number:
  US 20170270086A1
• Time in Patent Office: 1,203 Days
• Field of Search: 715271
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

An apparatus for correcting a character string in a text of an embodiment includes a first converter, a first output unit, a second converter, an estimation unit, and a second output unit. The first converter recognizes a first speech of a first speaker, and converts the first speech to a first text. The first output unit outputs a first caption image indicating the first text. The second converter recognizes a second speech of a second speaker for correcting a character string to be corrected in the first text, and converts the second speech to a second text. The estimation unit estimates the character string to be corrected, based on text matching between the first text and the second text. The second output unit outputs a second caption image indicating that the character string to be corrected is to be replaced with the second text.

Claims

1. An apparatus for correcting a character string in a text, the apparatus comprising:

a hardware processor electrically coupled to a memory, and configured to;

acquire a first audio of a first speech of a first speaker;

convert the first audio to a first text;

output a first caption image of the first text;

acquire a second audio of a second speech of a second speaker for correcting a character string that is included in the first text;

convert the second audio to a second text;

search the first text for one or more similar blocks that are similar to the second text, wherein each block of the one or more similar blocks comprises at least a portion of a character string that matches the second text;

calculate, for each of the one or more similar blocks, a measure of text similarity between the similar block and the second text;

calculate, for each of the one or more similar blocks, a measure of acoustic similarity by performing acoustic matching between comparison sections of the first audio corresponding to the one or more similar blocks and the second audio;

determine an estimated similarity for each of the one or more similar blocks by at least combining the measure of text similarity and the measure of acoustic similarity in a comparison section that corresponds to the similar block;

determine that a character string in a similar block of the one or more similar blocks is the character string to be corrected, wherein the estimated similarity associated with the similar block is equal to or more than a threshold; and

output a second caption image indicating the first text, the second text, and a position of the character string to be corrected in the first text, and indicating that the character string to be corrected is to be replaced with the second text.

2. The apparatus according to claim 1, wherein

when the first text includes a plurality of character strings in which the similarity with the second text is equal to or more than the threshold, the hardware processor estimates each of the character strings as a candidate character string to be corrected;

and

the hardware processor outputs a selection image for causing the second speaker to select, from the candidate character strings to be corrected in the first text, the candidate character string to be corrected that is to be replaced with the second text, and wherein

the hardware processor is further configured to receive selection input of selecting the candidate character string to be corrected that is to be replaced with the second text in the selection image, wherein

the hardware processor estimates the candidate character string to be corrected that is selected from among the candidate character strings to be corrected by the selection input as the character string to be corrected; and

the hardware processor outputs the second caption image indicating that the character string to be corrected in the first text is to be replaced with the second text.

3. The apparatus according to claim 2, wherein

the selection input corresponds to a touch input of touching a position on the candidate character string to be corrected or a vicinity position of the candidate character string to be corrected that is to be replaced with the second text on the selection image, and

the hardware processor estimates the candidate character string to be corrected, from among the candidate character strings to be corrected, that is closer to the position touched by the selection input as the character string to be corrected.

4. The apparatus according to claim 2, wherein the selection input corresponds to a touch input of selecting the candidate character string to be corrected, by touching a position on the candidate character string to be corrected or a vicinity position of the candidate character string to be corrected that is to be replaced with the second text, on the selection image.

5. The apparatus according to claim 2, wherein

the selection input is input of moving the second text to a position on the candidate character string to be corrected or a vicinity position of the candidate character string to be corrected that is to be replaced with the second text, and

the hardware processor estimates the candidate character string to be corrected, among the candidate character strings to be corrected, that is closer to the position to which the second text is moved by the selection input as the character string to be corrected.

6. The apparatus according to claim 1, wherein

the hardware processor is further configured to receive selection input of selecting the character string that is to be replaced with the second text on the first caption image, and

the hardware processor estimates the character string selected by the selection input as the character string to be corrected when the first text includes a plurality of character strings in which the similarity with the second text is equal to or more than the threshold.

7. The apparatus according to claim 5, wherein

the selection input corresponds to a touch input of touching a position on the character string or a vicinity position of the character string that is to be replaced with the second text on the first caption image, and

the hardware processor estimates the character string that is closer to the position touched by the selection input, among the character strings in which the similarity with the second text is equal to or more than the threshold, as the character string to be corrected.

8. The apparatus according to claim 5, wherein the selection input corresponds to a touch input of selecting the character string by touching a position on the character string or a vicinity position of the character string that is to be replaced with the second text on the first caption image.

9. The apparatus according to claim 1, wherein the second speech is shorter than the first speech.

10. A method for correcting a character string in a text, the method comprising:

acquiring a first audio of a first speech of a first speaker;

converting the first audio to a first text;

outputting a first caption image of the first text;

acquiring a second audio of a second speech of a second speaker for correcting a character string that is included in the first text;

converting the second audio to a second text;

searching the first text for one or more similar blocks that are similar to the second text, wherein each block of the one or more similar blocks comprises at least a portion of a character string that matches the second text;

calculating, for each of the one or more similar blocks, a measure of text similarity between the similar block and the second text;

calculating, for each of the one or more similar blocks, a measure of acoustic similarity by performing acoustic matching between comparison sections of the first audio corresponding to the one or more similar blocks and the second audio;

determining an estimated similarity for each of the one or more similar blocks by at least combining the measure of text similarity and the measure of acoustic similarity in a comparison section that corresponds to the similar block;

determine that a character string in a similar block of the one or more similar blocks is the character string to be corrected, wherein the estimated similarity associated with the similar block is equal to or more than a threshold; and

outputting a second caption image indicating the first text, the second text, and a position of the character string to be corrected in the first text, and indicating that the character string to be corrected is to be replaced with the second text.

11. A computer program product having a non-transitory computer readable medium including programmed instructions that cause a computer to execute:

acquiring a first audio of a first speech of a first speaker;

converting the first audio to a first text;

outputting a first caption image of the first text;

acquiring a second audio of a second speech of a second speaker for correcting a character string that is included in the first speech;

converting the second audio to a second text;

searching the first text for one or more similar blocks that are similar to the second text, wherein each block of the one or more similar blocks comprises at least a portion of a character string that matches or is close to the second text;

calculating, for each of the one or more similar blocks, a measure of text similarity between the similar block and the second text;

calculating, for each of the one or more similar blocks, a measure of acoustic similarity by performing acoustic matching between comparison sections of the first audio corresponding to the one or more similar blocks and the second audio;

determining an estimated similarity for each of the one or more similar blocks by at least combining the measure of text similarity and the measure of acoustic similarity in a comparison section that corresponds to the similar block;

determining that a character string in a similar block of the one or more similar blocks is the character string to be corrected, wherein the estimated similarity associated with the similar block is equal to or more than a threshold; and

outputting a second caption image indicating the first text, the second text, and a position of the character string to be corrected in the first text, and indicating that the character string to be corrected is to be replaced with the second text.