System and method for adapting speech playback speed to typing speed

• US 6,952,673 B2
• Filed: 02/20/2001
• Issued: 10/04/2005
• Est. Priority Date: 02/20/2001
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Amir, Arnon, Rodeh, Michael
• Primary Examiner(s): McFadden, Susan
• Assistant Examiner(s): Vo, Huyen X.
• Application Number:
  US09/789,452
• Publication Number:
  US 20020116188A1
• Time in Patent Office: 1,687 Days
• Field of Search: 704/211, 704/207, 704/235, 704/272, 704/231, 704/201, 704/260, 179/100.1 R, 179/100.1 VC, 341/22, 715/534
• US Class Current: 704/235
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

A system and method for automatically adjusting the rate at which recorded speech is played back as a typist manually transcribes the speech. The typing speed is measured and a speech playback rate determined based on the measured speed. The playback rate of the audio is then automatically increased or decreased as appropriate to match the typing speed.

Claims

1. A computer-implemented method for facilitating efficient transcription of audible speech from an audio system, comprising:

measuring a typing speed;

generating a signal based on the typing speed;

using the signal to establish a rate at which the audible speech is played by the audio system; and

assigning each word in recognized speech with a respective expected typing duration by a predetermined user, for facilitating estimating the typing rate.

2. The method of claim 1, wherein the signal represents a playback rate correction.

3. The method of claim 1, wherein the act of establishing the rate includes:

detecting a user-initiated pause in the audio system, and in response thereto reducing a playback rate; and

/or

detecting a continuous period of typing at least a first predetermined time period in length characterized by having pause periods all less than a second predetermined time period, and in response increasing the playback rate.

4. The method of claim 1, wherein the act of establishing the rate includes:

determining a number of words typed per a unit time, and establishing a playback rate based thereon.

5. The method of claim 1, wherein the act of establishing the rate includes:

determining a number of characters typed per a unit time, and establishing a playback rate based thereon.

6. The method of claim 1, wherein the act of establishing the rate includes:

determining a number of phonemes typed per a unit time, and establishing a playback rate based thereon.

7. The method of claim 1, further comprising reading ahead audio before it is played, such that the act of establishing the rate at which the audible speech is played is undertaken before the speech is played.

8. The method of claim 1, further comprising detecting a typing pause having at least a predetermined duration, and automatically stopping playback of the audio in response thereto.

9. The method of claim 1, further comprising:

detecting a stroke of a predetermined key; and

causing the audio system to replay audio in response to the stroke.

10. The method of claim 1, wherein the rate at which the audible speech is played is established using at least one of:

time scale modification (TSM), inserting pauses between words, inserting pauses between sentences.

11. The method of claim 1, wherein the audible speech is part of an audio stream, and the method includes skipping non-speech parts of the audio stream.

12. The method of claim 1, further comprising using speech recognition for automatic error detection and/or word completion.

13. The method of claim 1, further comprising detecting a change of speaker making the audible speech, and marking the change in response.

14. A computer program product to undertake logic for dynamically establishing a playback rate of an audio system, the logic including:

logic means for receiving manual input representing a transcription of audio having a playback rate;

logic means for determining a typing speed based on the means for receiving;

logic means for using the typing speed to establish a playback rate; and

assigning each word in recognized speech with a respective expected typing duration by a predetermined user, for facilitating estimating the typing rate.

15. The computer program product of claim 14, further comprising:

logic means for detecting a user-initiated pause in the audio system, and in response thereto reducing the playback rate; and

/or

logic means for detecting a continuous period of typing at least a first predetermined time period in length characterized by having pause periods all less than a second predetermined time period, and in response increasing the playback rate.

16. The computer program product of claim 14, wherein the means for using includes:

logic means for determining a number of words typed per a unit time, and establishing the playback rate based thereon.

17. The computer program product of claim 14, wherein the means for using includes:

logic means for determining a number of characters or phonemes typed per a unit time, and establishing the playback rate based thereon.

18. The computer program product of claim 14, further comprising logic means for reading ahead audio before it is played.

19. The computer program product of claim 14, further comprising logic means for detecting a typing pause having at least a predetermined duration, and automatically stopping playback of the audio in response thereto.

20. The computer program product of claim 14, further comprising:

logic means for detecting a stroke of a predetermined key; and

logic means for causing the audio system to replay audio in response to the stroke.

21. The computer program product of claim 14, wherein the audio is part of an audio stream, and the product includes logic means for skipping non-speech parts of the audio stream.

22. The computer program product of claim 14, further comprising logic means for using speech recognition for automatic error detection and/or word completion.

23. The computer program product of claim 14, further comprising logic means for detecting a change of speaker making the audio, and marking the change in response.

24. An audio transcription computer system, comprising:

at least one computer including a module having logical structure to determine typing speed;

at least one audio system receiving feedback representative of typing speed from the computer and in response applying an audio playback rate to audio; and

assigning each word in recognized speech with a respective expected typing duration by a predetermined user, for facilitating estimating the typing rate.

25. The system of claim 24, wherein the audio system includes at least one time scale modification device applying the playback rate to audio, and the feedback from the module establishes the playback rate.

26. The system of claim 25, wherein the module determines typing speed at least in part by:

detecting a user-initiated pause in the audio system, and in response thereto reducing the playback rate; and

/or

detecting a continuous period of typing at least a first predetermined time period in length characterized by having pause periods all less than a second predetermined time period, and in response increasing the playback rate.

27. The system of claim 25, wherein the module determines typing speed at least in part by determining a number of words or phonemes or characters typed per a unit time, and establishing the playback rate based thereon.

28. The system of claim 25, wherein the time scale modification device applies the playback rate to read ahead audio.

29. The system of claim 25, wherein the computer detects a typing pause having at least a predetermined duration, and causes the audio system to automatically stop playback of the audio in response thereto.

30. The system of claim 25, wherein the computer detects a stroke of a predetermined key, and causes the audio system to replay audio in response thereto.

31. The system of claim 25, further comprising a speech recognition module for undertaking at least one of:

automatic word completion based on a prefix already typed and a speech recognition result, and determining between one of several alternative interpretations of an audio segment based on a corresponding transcript.

32. The system of claim 24, wherein the audio is part of an audio stream, and the module skips non-speech parts of the audio stream.

33. The system of claim 24, wherein the module uses speech recognition for automatic error detection and/or word completion.

34. The system of claim 24, wherein the module detects a change of speaker making the audio, and marks the change in response.

35. A computer-implemented method for facilitating efficient transcription of audible speech from an audio system, comprising:

measuring a typing rate;

measuring a speech rate;

generating a signal based on the typing rate and speech rate;

using the signal to establish a rate at which the audible speech is played by the audio system; and

assigning each word in recognized speech with a respective expected typing duration by a predetermined user, for facilitating estimating the typing rate.

36. The method of claim 35, wherein the signal represents a playback rate correction.

37. The method of claim 35, wherein the act of establishing the typing rate includes:

detecting a user-initiated pause in the audio system, and in response thereto reducing a playback rate; and

/or

detecting a continuous period of typing at least a first predetermined time period in length characterized by having pause periods all less than a second predetermined time period, and in response increasing the playback rate.

38. The method of claim 35, wherein the act of establishing the typing rate includes:

determining a number of words typed per a unit time, and establishing a playback rate based thereon.

39. The method of claim 35, wherein the act of establishing the typing rate includes:

determining a number of characters typed per a unit time, and establishing a playback rate based thereon.

40. The method of claim 35, wherein the act of establishing the typing rate includes:

determining a number of phonemes typed per a unit time, and establishing a playback rate based thereon.

41. The method of claim 35, further comprising reading ahead audio before it is played, such that the act of establishing the rate at which the audible speech is played is undertaken before the speech is played.

42. The method of claim 35, further comprising detecting a typing pause having at least a predetermined duration, and automatically stopping playback of the audio in response thereto.

43. The method of claim 35, further comprising:

detecting a stroke of a predetermined key; and

causing the audio system to replay audio in response to the stroke.

44. The method of claim 35, wherein the audible speech is part of an audio stream, and the method includes skipping non-speech parts of the audio stream.

45. The method of claim 35, further comprising using speech recognition for automatic error detection and/or word completion.

46. The method of claim 35, further comprising detecting a change of speaker making the audible speech, and marking the change in response.

47. The method of claim 35, further comprising using speech recognition to determine words spoken per second, or phonemes spoken per second, or characters spoken per second, or approximations thereof.

48. A computer system for facilitating efficient transcription of audible speech from an audio system, comprising:

means for measuring a typing rate;

means for measuring a speech rate;

means for generating a signal based on the typing rate and speech rate;

means for using the signal to establish a rate at which the audible speech is played by the audio system; and

assigning each word in recognized speech with a respective expected typing duration by a predetermined user, for facilitating estimating the typing rate.

49. The system of claim 48, wherein the signal represents a playback rate correction.

50. The system of claim 48, further comprising:

means for detecting a user-initiated pause in the audio system, and in response thereto reducing a playback rate; and

/or

means for detecting a continuous period of typing at least a first predetermined time period in length characterized by having pause periods all less than a second predetermined time period, and in response increasing the playback rate.

51. The system of claim 48, further comprising:

means for determining a number of words typed per a unit time, and establishing a playback rate based thereon.

52. The system of claim 48, further comprising:

means for determining a number of characters typed per a unit time, and establishing a playback rate based thereon.

53. The system of claim 48, further comprising:

means for determining a number of phonemes typed per a unit time, and establishing a playback rate based thereon.

54. The system of claim 48, further comprising means for reading ahead audio before it is played, such that the means for establishing the rate at which the audible speech is played is executed before the speech is played.

55. The system of claim 48, further comprising means for detecting a typing pause having at least a predetermined duration, and automatically stopping playback of the audio in response thereto.

56. The system of claim 48, further comprising:

means for detecting a stroke of a predetermined key; and

means for causing the audio system to replay audio in response to the stroke.

57. The system of claim 48, wherein the audible speech is part of an audio stream, and the system includes means for skipping non-speech parts of the audio stream.

58. The system of claim 48, further comprising means for using speech recognition for automatic error detection and/or word completion.

59. The system of claim 48, further comprising means for detecting a change of speaker making the audible speech, and marking the change in response.

60. The system of claim 48, further comprising means for using speech recognition to determine words spoken per second, or phonemes spoken per second, or characters spoken per second, or approximations thereof.