Word tagging and editing system for speech recognition

• US 5,960,447 A
• Filed: 11/13/1995
• Issued: 09/28/1999
• Est. Priority Date: 11/13/1995
• Status: Expired due to Term

• Current Assignee: Advanced Voice Recognition Systems, Inc.
• Original Assignee: Douglas Holt, Joseph H Miglietta, Michael K Davis
• Inventors: Holt, Douglas, Davis, Michael K, Miglietta, Joseph H
• Primary Examiner(s): Feild, Joseph H.
• Application Number:
  US08/556,077
• Time in Patent Office: 1,415 Days
• Field of Search: 395/762, 395/777, 395/802, 395/793, 395/2.44, 395/2.6, 395/2.87, 707/501, 707/515, 707/531, 707/513, 345/302, 704/235, 704/270
• US Class Current: 715/201
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A word tagging and editing system for speech recognition receives recognized speech text from a speech recognition engine, and creates tagging information that follows the speech text as it is received by a word processing program or other program. The body of text to be edited in connection with the word processing program may be selected and cut and pasted and otherwise manipulated, and the tags follow the speech text. A word may be selected by a user, and the tag information used to point to a sound bite within the audio data file created initially by the speech recognition engine. The sound bite may be replayed to the user through a speaker. The practical results include that the user may confirm the correctness of a particular recognized word, in real time whilst editing text in the word processor. If the recognition is manually corrected, the correction information may be supplied to the engine for use in updating a user profile for the user who dictated the audio that was recognized. Particular tagging approaches are employed depending on the particular word processor being used.

Claims

1. An apparatus for editing text resulting from speech recognition, said apparatus comprising an audio file, a word processing file, a word processor, and a playback facility, the playback facility further comprising a speaker:

the audio file comprising audio information;

the word processing file comprising text words and tag information for each of the text words, the tag information linking text words to respective audio information;

the word processor having user inputs and operatively coupled with the word processing file and disposed to select ones of the text words and tag information at a first location;

the word processor responsive to a first user input for copying selected text words and tag information, and responsive to a second user input for inserting said selected text words and tag information into the word processing file at a second location differing from the first location;

the word processor further responsive to a third user input for selecting ones of the text words and tag information at the second location;

the apparatus further comprising a playback facility responsive to a fourth user input and to the selected tag information at the second location, for identifying audio information in the audio file linked to the text words selected at the second location, and for playing back the audio information via the speaker.

2. The apparatus of claim 1 in which the tag information comprises bookmarks, each bookmark comprising a unique identifier, a portion of the identifier defining the link to respective audio information, each bookmark disposed to begin after the first character of its corresponding word, and disposed to end after the last character of its corresponding word.

3. The apparatus of claim 1 in which the tag information comprises third party code, each item of third party code comprising information about the length of the respective dictated unit of text, information defining the link to respective audio information, and information defining the length of the item of third party code.

4. The apparatus of claim 1 in which the tag information comprises a bookmark, with a first character of the bookmark identifying the size of the string for the intended position of the bookmark.

5. The apparatus of claim 1 in which the word processor comprises a means for storing hidden text in the word processor file, and in which the tag information comprises hidden text.

6. The apparatus of claim 1 in which the word processor comprises a means for manipulating OLE objects, and in which the text words and respective tag information together comprise OLE objects.

7. The apparatus of claim 1 further comprising a microphone and a speech recognition engine, the speech recognition engine operatively coupled with the microphone and operatively coupled with the audio file for receiving speech at the microphone and storing audio information indicative of the speech in the audio file, the speech recognition engine further operatively coupled with the word processor for provision of recognized text words from the speech recognition to the word processor.

8. The apparatus of claim 7 in which the speech recognition engine further comprises a user profile file storing information relating to a user'"'"'s speech, the user profile used by the speech recognition engine for recognition of speech by the user, and in which the apparatus further comprises updating means responsive to a fifth user input for updating the user profile file subsequent to playback of the audio information via the speaker.

9. An apparatus for editing text resulting from speech recognition, said apparatus comprising an audio file, a word processing file, a word processor, and a playback facility, the playback facility further comprising a speaker:

the audio file comprising audio information;

the word processing file comprising text words and tag information for each of the text words, the tag information linking text words to respective audio information;

the word processor having user inputs and operatively coupled with the word processing file and disposed to select ones of the text words and tag information at a first location;

the word processor responsive to a first user input for copying and deleting selected text words and tag information, and responsive to a second user input for inserting said selected text words and tag information into the word processing file at a second location differing from the first location;

the word processor further responsive to a third user input for selecting ones of the text words and tag information at the second location;

the apparatus further comprising a playback facility responsive to a fourth user input and to the selected tag information at the second location, for identifying audio information in the audio file linked to the text words selected at the second location, and for playing back the audio information via the speaker.

10. The apparatus of claim 9 in which the tag information comprises bookmarks, each bookmark comprising a unique identifier, a portion of the identifier defining the link to respective audio information, each bookmark disposed to begin after the first character of its corresponding word, and disposed to end after the last character of its corresponding word.

11. The apparatus of claim 9 in which the tag information comprises third party code, each item of third party code comprising information about the length of the respective dictated unit of text, information defining the link to respective audio information, and information defining the length of the item of third party code.

12. The apparatus of claim 9 in which the tag information comprises a bookmark, with a first character of the bookmark identifying the size of the string for the intended position of the bookmark.

13. The apparatus of claim 9 in which the word processor comprises a means for storing hidden text in the word processor file, and in which the tag information comprises hidden text.

14. The apparatus of claim 9 in which the word processor comprises a means for manipulating OLE objects, and in which the text words and respective tag information together comprise OLE objects.

15. The apparatus of claim 9 further comprising a microphone and a speech recognition engine, the speech recognition engine operatively coupled with the microphone and operatively coupled with the audio file for receiving speech at the microphone and storing audio information indicative of the speech in the audio file, the speech recognition engine further operatively coupled with the word processor for provision of recognized text words from the speech recognition to the word processor.

16. The apparatus of claim 15 in which the speech recognition engine further comprises a user profile file storing information relating to a user'"'"'s speech, the user profile used by the speech recognition engine for recognition of speech by the user, and in which the apparatus further comprises updating means responsive to a fifth user input for updating the user profile file subsequent to playback of the audio information via the speaker.

17. An apparatus for editing text resulting from speech recognition, said apparatus comprising an audio file, a word processing file, a mirror file, a word processor, a mirror application, and a playback facility, the playback facility further comprising a speaker:

the audio file comprising audio information;

the word processing file comprising text words;

the mirror file comprising tag information for each of the text words, the tag information linking text words to respective audio information;

the word processor having user inputs and operatively coupled with the word processing file and disposed to select ones of the text words at a first location;

the word processor responsive to a first user input for copying selected text words, and responsive to a second user input for inserting said selected text words into the word processing file at a second location differing from the first location;

the mirror application responsive to the user inputs and operatively coupled with the mirror file and disposed to select ones of the tag information corresponding to the text words at the first location, the mirror application responsive to the first user input for copying tag information corresponding to the selected text words, and responsive to the second user input for inserting said tag information into the mirror file at a second location differing from the first location;

the word processor further responsive to a third user input for selecting ones of the text words at the second location;

the mirror application responsive to the third user input for selecting tag information corresponding to the ones of the text words at the second location;

the apparatus further comprising a playback facility responsive to a fourth user input and to the selected tag information at the second location, for identifying audio information in the audio file linked to the text words selected at the second location, and for playing back the audio information via the speaker.

18. The apparatus of claim 17 further comprising a microphone and a speech recognition engine, the speech recognition engine operatively coupled with the microphone and operatively coupled with the audio file for receiving speech at the microphone and storing audio information indicative of the speech in the audio file, the speech recognition engine further operatively coupled with the word processor for provision of recognized text words from the speech recognition to the word processor.

19. An apparatus for editing text resulting from speech recognition, said apparatus comprising an audio file, a word processing file, a mirror file, a word processor, a mirror application, and a playback facility, the playback facility further comprising a speaker:

the audio file comprising audio information;

the word processing file comprising text words;

the mirror file comprising tag information for each of the text words, the tag information linking text words to respective audio information;

the word processor having user inputs and operatively coupled with the word processing file and disposed to select ones of the text words at a first location;

the word processor responsive to a first user input for copying and deleting selected text words, and responsive to a second user input for inserting said selected text words into the word processing file at a second location differing from the first location;

the mirror application responsive to the user inputs and operatively coupled with the mirror file and disposed to select ones of the tag information corresponding to the text words at the first location, the mirror application responsive to the first user input for copying and deleting tag information corresponding to the selected text words, and responsive to the second user input for inserting said tag information into the mirror file at a second location differing from the first location;

the word processor further responsive to a third user input for selecting ones of the text words at the second location;

the mirror application responsive to the third user input for selecting tag information corresponding to the ones of the text words at the second location;

the apparatus further comprising a playback facility responsive to a fourth user input and to the selected tag information at the second location, for identifying audio information in the audio file linked to the text words selected at the second location, and for playing back the audio information via the speaker.

20. The apparatus of claim 19 further comprising a microphone and a speech recognition engine, the speech recognition engine operatively coupled with the microphone and operatively coupled with the audio file for receiving speech at the microphone and storing audio information indicative of the speech in the audio file, the speech recognition engine further operatively coupled with the word processor for provision of recognized text words frond the speech recognition engine to the word processor.

21. A method of editing text resulting from speech recognition for use with an audio file, a word processing file, a word processor, a playback facility, and a speaker, the method comprising the steps of:

establishing, within the word processing file, text words and tag information for each of the text words, the tag information linking text words to respective audio information;

selecting ones of the text words and tag information at a first location;

copying the selected text words and tag information;

inserting said selected text words and tag information into the word processing file at a second location differing from the first location;

selecting ones of the text words and tag information at the second location;

identifying audio information in the audio file linked to the text words selected at the second location; and

playing back the audio information via the speaker.

22. The method of claim 21 in which the tag information comprises bookmarks, each bookmark comprising a unique identifier, a portion of the identifier defining the link to respective audio information, each bookmark disposed to begin after the first character of its corresponding word, and disposed to end after the last character of its corresponding word.

23. The method of claim 21 in which the tag information comprises third party code, each item of third party code comprising information about the length of the respective dictated unit of text, information defining the link to respective audio information, and information defining the length of the item of third party code.

24. The method of claim 21 in which the tag information comprises a bookmark, with a first character of the bookmark identifying the size of the string for the intended position of the bookmark.

25. The method of claim 21 in which the word processor comprises a means for storing hidden text in the word processor file, and in which the tag information comprises hidden text.

26. The method of claim 21 in which the word processor comprises a means for manipulating OLE objects, and in which the text words and respective tag information together comprise OLE objects.

27. The method of claim 21 further characterized as being for use with a microphone and a speech recognition engine, further comprising the steps, performed before the establishing step, of:

receiving speech at the microphone;

storing audio information indicative of the speech in the audio file;

recognizing text words; and

storing the recognized words in the word processor file.

28. The method of claim 27 further characterized as being for use with a speech recognition engine further comprising a user profile file storing information relating to a user'"'"'s speech, the method further characterized in that the recognizing step uses information stored in the user profile file;

the method further comprising the step of updating the user profile file subsequent to playback of the audio information via the speaker.

29. A method of editing text resulting from speech recognition for use with an audio file, a word processing file, a word processor, a playback facility, and a speaker, the method comprising the steps of:

establishing, within the word processing file, text words and tag information for each of the text words, the tag information linking text words to respective audio information;

selecting ones of the text words and tag information at a first location;

copying the selected text words and tag information;

deleting the selected text words and tag information at the first location;

inserting said selected text words and tag information into the word processing file at a second location differing from the first location;

selecting ones of the text words and tag information at the second location;

identifying audio information in the audio file linked to the text words selected at the second location; and

playing back the audio information via the speaker.

30. The method of claim 29 in which the tag information comprises bookmarks, each bookmark comprising a unique identifier, a portion of the identifier defining the link to respective audio information, each bookmark disposed to begin after the first character of its corresponding word, and disposed to end after the last character of its corresponding word.

31. The method of claim 29 in which the tag information comprises third party code, each item of third party code comprising information about the length of the respective dictated unit of text, information defining the link to respective audio information, and information defining the length of the item of third party code.

32. The method of claim 29 in which the tag information comprises a bookmark, with a first character of the bookmark identifying the size of the string for the intended position of the bookmark.

33. The method of claim 29 in which the word processor comprises a means for storing hidden text in the word processor file, and in which the tag information comprises hidden text.

34. The method of claim 29 in which the word processor comprises a means for manipulating OLE objects, and in which the text words and respective tag information together comprise OLE objects.

35. The method of claim 30 further characterized as being for use with a microphone and a speech recognition engine, further comprising the steps, performed before the establishing step, of:

receiving speech at the microphone;

storing audio information indicative of the speech in the audio file;

recognizing text words; and

storing the recognized words in the word processor file.

36. The method of claim 35 further characterized as being for use with a speech recognition engine further comprising a user profile file storing information relating to a user'"'"'s speech, the method further characterized in that the recognizing step uses information stored in the user profile file;

the method further comprising the step of updating the user profile file subsequent to playback of the audio information via the speaker.

37. A method for editing text resulting from speech recognition, the method used with an audio file, a word processing file, a mirror file, a word processor, a mirror application, a playback facility, and a speaker, the method comprising the steps of:

storing recognized text in the word processing file;

storing in the mirror file tag information for each of the text words, the tag information linking text words to respective audio information;

selecting ones of the text words at a first location;

copying the selected text words;

inserting said selected text words into the word processing file at a second location differing from the first location;

select ones of the tag information corresponding to the text words at the first location;

copying tag information corresponding to the selected text words;

inserting said tag information into the mirror file at a second location differing from the first location;

selecting ones of the text words at the second location;

selecting tag information corresponding to the ones of the text words at the second location;

identifying audio information in the audio file linked to the text words selected at the second location; and

playing back the audio information via the speaker.

38. The method of claim 37 further characterized as being used with a microphone and a speech recognition engine, the method comprising the additional steps, performed before the first storing step, of:

receiving speech at the microphone;

storing audio information indicative of the speech in the audio file;

recognizing text words from the information in the audio file; and

providing recognized text words to the word processor.

39. A method for editing text resulting from speech recognition, the method used with an audio file, a word processing file, a mirror file, a word processor, a mirror application, a playback facility, and a speaker, the method comprising the steps of:

storing recognized text in the word processing file;

storing in the mirror file tag information for each of the text words, the tag information linking text words to respective audio information;

selecting ones of the text words at a first location;

copying the selected text words;

deleting the selected text words;

inserting said selected text words into the word processing file at a second location differing from the first location;

selecting ones of the tag information corresponding to the text words at the first location;

copying the tag information corresponding to the text words at the first location;

deleting the tag information corresponding to the text words at the first location;

inserting said tag information into the mirror file at a second location differing from the first location;

selecting ones of the text words at the second location;

selecting tag information corresponding to the ones of the text words at the second location;

identifying audio information in the audio file linked to the text words selected at the second location; and

playing back the audio information via the speaker.

40. The method of claim 39 further characterized as being used with a microphone and a speech recognition engine, the method comprising the additional steps, performed before the first storing step, of:

receiving speech at the microphone;

storing audio information indicative of the speech in the audio file;

recognizing text words from the information in the audio file; and

providing recognized text words to the word processor.

41. A speech recognition method for use with a speech recognition engine having an enrollment mode and a recognition mode and with a digital tape recorder having a microphone, the method comprising the steps of:

operatively coupling the digital tape recorder with the engine;

configuring the digital tape recorder so that audio information received at its microphone is communicated via the operative coupling to the engine;

performing the enrollment;

disconnecting the recorder from the engine;

dictating into the microphone of the recorder thereby recording dictated audio information therein;

operatively coupling the recorder with the engine;

playing back the audio information from the recorder and communicating said information via the operative coupling to the engine; and

recognizing the information within the engine.

42. The method of claim 41 wherein the recorder further comprises an automatic gain control circuit operatively coupled with the microphone whereby the information recorded and communicated external to the recorder is at a substantially constant signal level.