System and method using feedback speech analysis for improving speaking ability

• US 8,756,057 B2
• Filed: 11/02/2005
• Issued: 06/17/2014
• Est. Priority Date: 11/02/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Miller, Steven Michael, Sand, Anne R.
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US11/265,591
• Publication Number:
  US 20070100626A1
• Time in Patent Office: 3,149 Days
• Field of Search: 704/270, 704/271, 704/235, 704/277, 704/270.1, 704/275, 704/273, 704/9, 704/246, 704/276, 704/278, 434/185, 434/186, 434/187, 434/169
• US Class Current: 704/235
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/10 Transforming into visible information

Abstract

A speech analysis system and method for analyzing speech. The system includes: a voice recognition system for converting inputted speech to text; an analytics system for generating feedback information by analyzing the inputted speech and text; and a feedback system for outputting the feedback information.

Claims

1. A speech analysis system for analyzing speech, comprising:

at least one hardware processor that;

converts inputted speech received from a speaker to text;

displays the text in a textual interface displayed on at least one display device;

automatically generates feedback information, that comprises results of an analysis of the inputted speech and/or text, by automatically analyzing the inputted speech and/or text; and

automatically outputs the feedback information as annotations in the textual interface, wherein the annotations are distinct from the inputted speech and text.

2. The speech analysis system of claim 1, wherein the at least one hardware processor generates feedback information, in part, by analyzing the text and determining a complexity level.

3. The speech analysis system of claim 1, wherein the at least one hardware processor generates feedback information, in part by capturing timing related information about the inputted speech.

4. The speech analysis system of claim 1, wherein the at least one hardware processor generates feedback information, in part, by capturing volume data for the inputted speech.

5. The speech analysis system of claim 1, wherein the at least one hardware processor generates feedback information, in part, by detecting the use of speech elements selected from the group consisting of:

improper words, improper grammar, stutter, mumbling, and nervousness.

6. The speech analysis system of claim 1, wherein the annotations include data that appears in a text window containing the textual information and data that appears in a second window adjacent the text window.

7. The speech analysis system of claim 1, wherein the at least one hardware processor displays, on the at least one display device, feedback information in real time as inputted speech is received.

8. The speech analysis system of claim 1, wherein the at least one hardware processor displays, on the at least one display device, feedback information based on a previously inputted speech.

9. The speech analysis system of claim 1, wherein the at least one hardware processor displays feedback in a display on the at least one display device, the display selected from the group consisting of:

a dashboard and a teleprompter.

10. The speech analysis system of claim 1, wherein the at least one hardware processor highlights words having associated annotations.

11. The speech analysis system of claim 1, further comprising a training system that provides a set of drills that can be used to improve speaking capabilities.

12. The speech analysis system of claim 1, further comprising a collaboration system that allows a third party collaborator to include additional feedback information in the textual interface.

13. A computer program product stored on a non-transitory tangible computer usable medium for analyzing speech, comprising:

program code that when executed by a processor;

converts inputted speech received from a speaker to text;

displays the text in a textual interface;

automatically generates feedback information, that comprises results of an analysis of the inputted speech and/or text, by automatically analyzing the inputted speech and/or text; and

automatically outputs the feedback information as annotations in the textual interface, wherein the annotations are distinct from the inputted speech and text.

14. The computer program product of claim 13, wherein generating feedback information includes capabilities selected from the group consisting of:

textual analytics, capturing timing related information about the inputted speech, and capturing volume data for the inputted speech.

15. The computer program product of claim 13, wherein generating feedback information further comprises detecting the use speech of elements selected from the group consisting of:

improper words, improper grammar, stutter, mumbling, and nervousness.

16. The computer program product of claim 13, wherein the annotations are displayed in a text window and a second window adjacent the text window.

17. The computer program product of claim 13, wherein the feedback information is displayed in real time as inputted speech is received.

18. The computer program product of claim 13, wherein the feedback information is displayed based on a previously inputted speech.

19. The computer program product of claim 13, wherein the feedback information is displayed within a display selected from the group consisting of:

a dashboard and a teleprompter.

20. The computer program product of claim 13, wherein the feedback information is displayed in the textual interface that displays highlighted text along with annotations.

21. The computer program product of claim 13, further comprising training means that provides a set of drills that can be used to improve speaking capabilities.

22. The computer program product of claim 13, further comprising collaboration means that allows a third party collaborator to input additional feedback information.

23. A method for analyzing speech, comprising:

converting inputted speech received from a speaker to text and displaying the text in a textual interface;

capturing timing related information and volume information from the speech;

analyzing the text to generate textual information;

automatically generating annotations for the text that comprise results of an analysis of the speech and/or the text and displaying the annotations in the textual interface, wherein the annotations are distinct from the inputted speech and text;

generating feedback information based on the capturing, analyzing, and generating steps; and

outputting the feedback information.

24. The method of claim 23, wherein the analyzing step detects the use of speech elements selected from the group consisting of:

improper words, improper grammar, stutter, mumbling, and nervousness.

25. The method of claim 23, wherein the feedback information is displayed in real time as inputted speech is received.

26. The method of claim 23, wherein the feedback information is displayed based on a previously inputted speech.

27. The method of claim 23, wherein the feedback information is outputted to a display selected from the group consisting of:

a dashboard and a teleprompter.

28. The method of claim 23, wherein the textual interface that displays annotations in a text window and in a second window adjacent the text window.

29. A method for deploying a speech analysis application, comprising:

providing a computer infrastructure being operable to;

convert inputted speech received from a speaker to text;

display the text in a textual interface;

automatically generate feedback information, that comprises an analysis of the inputted speech and/or the text, by automatically analyzing the inputted speech and/or text; and

automatically output the feedback information as annotations in the textual interface, wherein the annotations are distinct from the inputted speech and text.