Recognizing speech, and processing data

• US 20050143994A1
• Filed: 11/30/2004
• Published: 06/30/2005
• Est. Priority Date: 12/03/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Mori, Shinsuke, Nishimura, Masafumi, Itoh, Nobuyasu
• Granted Patent: US 8,150,687 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An example embodiment of the invention includes a speech recognition processing unit for specifying speech segments for speech data, recognizing a speech in each of the speech segments, and associating a character string of obtained recognition data with the speech data for each speech segment, based on information on a time of the speech, and an output control unit for displaying/outputting the text prepared by sorting the recognition data in each speech segment. Sometimes, the system further includes a text editing unit for editing the prepared text, and a speech correspondence estimation unit for associating a character string in the edited text with the speech data by using a technique of dynamic programming.

Claims

1. A system for recognizing speech, comprising:

a microphone for acquiring speech data;

a speech recognition processing unit for specifying speech segments for the speech data acquired by the microphone, recognizing a speech in each of the speech segments, and associating a character string of obtained recognition data with the speech data for each speech segment, based on information on a time of the speech;

a text preparation unit for preparing text from the recognition data in each speech segment obtained through the processing by the speech recognition processing unit; and

an output control unit for displaying/outputting the text prepared by the text preparation unit.

2. The system for recognizing speech according to claim 1, wherein the output control unit reproduces the speech data corresponding to the designated recognition data in the text, the recognition data being designated using the speech segment as a unit.

3. The system for recognizing speech according to claim 1,

wherein a plurality of the microphones are prepared in accordance with the number of speakers, and

the speech recognition processing unit processes speeches acquired by the plurality of microphones for each channel and, to the recognition data in each channel, imparts identification information for identifying a speaker associated with the channel.

4. The system for recognizing speech according to claim 3, wherein the output control unit retrieves, from the text, the recognition data of the same speaker specified by the identification information, and reproduces the speech data corresponding to the retrieved recognition data.

5. The system for recognizing speech according to claim 1, wherein the output control unit retrieves, from the text, the recognition data including a specific phrase, and reproduces the speech data corresponding to the retrieved recognition data.

6. The system for recognizing speech according to claim 1, further comprising:

a text editing unit for editing the text prepared by the text preparation unit; and

a speech correspondence estimation unit for associating a character string in the text edited by the text editing unit with the speech data,

wherein the output control unit displays/outputs the text edited by the text editing unit.

7. The system for recognizing speech according to claim 6, wherein the speech correspondence estimation unit matches pieces of the text before and after editing by the text editing unit with each other by means of dynamic programming to estimate a correspondence between both the pieces of the text and, based on a result of the estimation, associates the speech data in association with the character string of the recognition data in the text before editing with the text after editing.

8. An apparatus for processing data, comprising:

editing means for receiving a character string and correspondence data describing a correspondence between the character string and speech data, and for editing the character string; and

speech correspondence estimating means for matching the character string edited by the editing means with the character string before editing by means of dynamic programming to estimate a correspondence between both the character strings, and for associating the edited character string with the speech data, based on a result of the estimation.

9. The apparatus for processing data according to claim 8, wherein, for operations of substitution, insertion and deletion of a character by the editing means, the speech correspondence estimating means defines a distance scale for evaluating a difference between patterns of the character strings before and after editing, and matches the character strings with each other.

10. The apparatus for processing data according to claim 8, wherein the speech correspondence estimating means matches the character strings with each other by using a local path constraint allowing a presence of a character which does not have a correspondence in the character strings before and after editing by the editing means.

11. The apparatus for processing data according to claim 8, further comprising speech recognition means for recording a speech to recognize the speech and for associating a character string of obtained recognition data with the speech data based on information on a time of the speech,

wherein the editing means receives, as the correspondence data, the recognition data associated with the speech data by the speech recognition means.

12. The apparatus for processing data according to claim 8, further comprising output controlling means for displaying/outputting the character string edited by the editing means, for accepting a designation of a range in the displayed character string and for reproducing the speech data associated with the specified range in the character string by the speech correspondence estimating means.

13. A data processing method for processing speech data by using a computer, the method comprising the steps of:

by the computer, specifying speech segments for speech data acquired by speech inputting means, and recognizing a speech in each of the speech segments;

by the computer, associating a character string of obtained recognition data with the speech data for each speech segment, based on information on a time of the speech; and

by the computer, preparing text from the obtained recognition data in each speech segment.

14. The data processing method according to claim 13, wherein the step of recognizing a speech includes the steps of:

recognizing speeches acquired by a plurality of microphones prepared in accordance with the number of speakers;

comparing speech powers in respective channels with one another on a time axis, and specifying speech segments for each of the channels; and

applying the specified speech segments to results of recognizing the speeches in the respective channels, specifying the recognition data for speeches of speakers corresponding to the respective channels, and imparting identification information of the speakers to the recognition data.

15. The data processing method according to claim 13, further comprising the steps of:

editing the text prepared by the step of preparing text; and

matching a character string in the edited text with the character string before editing by means of dynamic programming to estimate a correspondence between both the character strings, and associating the edited character string with the speech data based on a result of the estimation.

24. A program for allowing a computer to execute processing corresponding to each of the steps in the method according to any one of claims 13 to 18.

16. A data processing method by a computer, comprising the steps of:

by the computer, receiving a character string and correspondence data describing a correspondence between the character string and speech data, and editing the character string; and

by the computer, matching the edited character string with the character string before editing by means of dynamic programming to estimate a correspondence between both the character strings, and associating the edited character string with the speech data based on a result of the estimation.

17. The data processing method according to claim 16, wherein, in the step of associating the character string with the speech data, for operations of substitution, insertion and deletion of a character in the editing of the character string, a distance scale for evaluating a difference between patterns of the character strings before and after editing is defined, and the character strings are matched with each other.

18. The data processing method according to claim 16, wherein, in the step of associating the character string with the speech data, the character strings before and after editing are matched with each other by using a local path constraint allowing a presence of a character which does not have a correspondence in the character strings before and after editing.

19. A program for allowing a computer to function as:

speech recognition processing means for specifying speech segments for speech data acquired by a microphone, recognizing a speech in each of the speech segments, and associating a character string of obtained recognition data with the speech data for each speech segment, based on information on a time of the speech;

text preparing means for preparing text from the obtained recognition data in each speech segment; and

output controlling means for displaying/outputting the prepared text.

20. The program according to claim 19, wherein, as a function of the output controlling means, the computer is allowed to reproduce the speech data corresponding to the designated recognition data in the text, the recognition data being designated using the speech segment as a unit.

21. The program according to claim 19,

wherein, as a function of the speech recognition processing means, the computer is allowed to process speeches acquired by a plurality of the microphones for each channel and to impart, to recognition data of each channel, identification information for identifying a speaker associated with the channel, and

as a function of the output controlling means, the computer is allowed to retrieve, from the text, the recognition data of the same speaker specified by the identification information, and to reproduce the speech data corresponding to the retrieved recognition data.

22. The program according to claim 19, wherein, as a function of the output controlling means, the computer is allowed to retrieve, from the text, the recognition data including a specific phrase, and to reproduce the speech data corresponding to the retrieved recognition data.

23. The program according to claim 19, further allowing the computer to function as:

text editing means for editing the text prepared by the function of the text preparing means; and

speech correspondence estimating means for matching a character string in the edited text with a character string before editing by means of dynamic programming to estimate a correspondence between both the character strings, and associating the edited character string with the speech data based on a result of the estimation.

25. A recording medium recording the program according to any one of claims 19 to 24 so that the program is computer-readable.

26. A system for preparing an electronic investigation, comprising:

a plurality of microphones prepared individually for attendants at a court;

speech recognition processing means for recognizing speeches acquired by the plurality of microphones for each channel, and comparing speech powers of the respective channels with one another on a time axis to specify speech segments in each channel, thus specifying recognition data in each of the speech segments in the speeches of speakers corresponding to the respective channels, imparting identification information of the speaker to the recognition data, and associating a character string of the recognition data with speech data, based on information on a time of the speech; and

outputting means for sorting the recognition data in each speech segment of each speaker based on the times of the speeches to prepare and display/output an electronic investigation, the recognition data being obtained through the processing by the speech recognition processing means, and reproducing the speech data corresponding to the designated recognition data in the electronic investigation, the recognition data being designated using the speech segment as a unit.

27. The system for preparing an electronic investigation according to claim 26, further comprising:

editing means for editing the prepared electronic investigation; and

speech correspondence estimating means for matching pieces of text before and after editing by the editing means by means of dynamic programming to estimate a correspondence between both the pieces of the text, and based on a result of the estimation, associating the speech data in association with the character string of the recognition data in the text before editing with the text after editing,

wherein the outputting means displays/outputs the electronic investigation edited by the editing means.