Computerized notetaking system and method

• US 20060075347A1
• Filed: 10/04/2005
• Published: 04/06/2006
• Est. Priority Date: 10/05/2004
• Status: Abandoned Application

• Current Assignee: Peter H. Rehm
• Original Assignee: Peter H. Rehm
• Inventors: Rehm, Peter H.
• Application Number:
  US11/243,589
• Publication Number:
  US 20060075347A1
• Time in Patent Office: Days
• US Class Current: 715/727
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals

Abstract

A computerized notetaking system that records audio and links notes to the audio and has enhancements such as: Always-on audio recording and external timestamp button that work even when the system is turned off. A “next-topic” command that prepares and timestamps a new paragraph while allowing the user to complete the current paragraph. Commands for creating callouts and sidebar boxes in the user notes. A choice of audio filters. Speech recognition for searching through and navigating an audio recording. Speech recognition accuracy enhancement based upon typed notes. Playback, including lockstep playback, that prefers starting and stopping at word boundaries without overlap when possible. A self-adjusting preplay parameter. A “repeat slower” command that replays a few seconds of audio and then resumes at normal speed. Integrated background audio that plays music or white noise when notes-related audio is not being played. Several other enhancements are disclosed.

Claims

1. A method of taking notes of an event on a computerized notetaking device that has a display and at least when needed access to a recording device at least when needed an audio output device, said method comprising the steps of:

(a) said recording device making an audio recording of said event;

(b) said computerized notetaking device inputting notes about said event from a user and inserting these notes into a document and displaying at least a portion of said document on said display and timestamping a plurality of these notes to synchronize them with said audio recording;

(c) said computerized notetaking device, having access to both the audio recording and the notes, inputting a timestamp-selective play command from a user and responsive to the play command, playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter; and

(d) said computerized notetaking device also performing at least one step selected from the group consisting of the following Arabic-numbered steps;

(1) said computerized notetaking device playing background audio when not playing said audio recording, and additionally responsive to said play command, interrupting the background audio while playing said audio recording;

(2) said computerized notetaking device filtering the recorded audio in the time domain by attempting to find a fundamental frequency that falls in the range of fundamental frequencies of human speech and upon finding any such fundamental frequency, attempting to extract the integral harmonics of this fundamental frequency, the integral harmonics following the fundamental frequency as it varies over time, and retaining only the fundamental frequency and its integral harmonics and short sounds having no fundamental frequency to make the voice of a particular speaker more understandable;

(3) said computerized notetaking device, prior to playing said audio recording starting from a particular point, searching the audio recording for a pause in the sound level of the audio recording that is near said particular point and upon finding such a pause further adjusting said particular point so play starts near the end of said pause that was found;

(4) said computerized notetaking device, when responding to a lockstep play command that is accompanied by a scheduled stop point, searching the audio recording for a pause in the sound level of the audio recording that is near the scheduled stop point and upon finding such a pause stopping audio playback in the pause that was found;

(5) said computerized notetaking device, while inputting notes to a first location, also inputting a next-topic command and responsive to said next-topic command preparing a second location for notes and establishing a timestamp link that does not link to the notes at the first location but rather links to the second location for notes, and after responding to said next-topic command continuing to input notes to the first location; and

(6) said computerized notetaking device, while inputting notes to a first location, also inputting a next-topic command and responsive to said next-topic command preparing a second location for notes and establishing a timestamp link that does not link to the notes at the first location but rather links to the second location for notes and prompting for a next-topic label, and after inputting a termination to the next-topic label displaying the next-topic label at the second location and continuing to input notes to the first location.

2. The method of claim 1 wherein said computerized notetaking device additionally performs a second step chosen from the Arabic-numbered steps, whereby the method comprises two of the Arabic-numbered steps.

3. The method of claim 2 wherein said computerized notetaking device additionally performs a third step chosen from the Arabic-numbered steps, whereby the method comprises three of the Arabic-numbered steps.

4. The method of claim 1 wherein said computerized notetaking device performs all of the Arabic-numbered steps.

5. The method of claim 1 wherein said step of playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter additionally comprises the steps of playing a predetermined amount of the audio recording at a predetermined maximum speed just prior to the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter and then transitioning to a regular playback speed when substantially at the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter.

6. The method of claim 1 wherein said step of inputting notes comprises inputting keystrokes from a keyboard.

7. The method of claim 1 wherein said step of inputting notes comprises inputting stylus strokes from a pointing device.

8. The method of claim 1—

additionally comprising the steps of inputting an insert sidebar box command and responsive to said insert sidebar box command, said computerized notetaking device creating a new text area that is inside said document and that has its own flow of text within said new text area that does not mix with any other flow of text of said document, and displaying said new text area in a box on said display, reflowing any existing notes around the box as necessary, and positioning the caret in the box.

10. The method of claim 1 additionally comprising the step of said computerized notetaking device attempting speech recognition of the audio recording and generating automatically recognized text that is also linked to said audio recording, and said computerized notetaking device displaying at least a portion of said automatically recognized text in manner that is distinguishable from said notes and said computerized notetaking device inputting a play command that is selective of a portion of said automatically recognized text and responsive to said play command, playing the portion of the audio recording that was linked to said selected portion of said automatically recognized text.

11. The method of claim 1 additionally comprising the step of, further responsive to said step of inputting a timestamp-selective play command, after said step of playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter, said computerized notetaking device keeping track of the next command and when 67% of the time or more the next command is a rewind command, gradually increasing said preplay parameter and when 67% of the time or more the next command is a fast forward command, gradually reducing said preplay parameter until a predetermined minimum preplay parameter is reached.

12. The method of claim 1 additionally comprising the step of during playing of audio, inputting movements of a scrolling wheel and responsive to these movements rewinding the audio playback for movements in a first direction and fast forwarding the audio playback for movements in a second direction.

13. The method of claim 1 additionally comprising the step of inputting a replay slower command and responsive to each said replay slower command, rewinding the audio by a preconfigured replay amount and reducing the playback speed to a preconfigured replay speed and repeating the just played audio at the preconfigured replay speed, and restoring the previous playback speed when replay is finished.

9. The method of claim L additionally comprising the steps of inputting a create callout command accompanied by a choice of target and responsive to said create callout command, said computerized notetaking device preparing an area for explanatory text, placing the caret inside the area for explanatory text and drawing connecting indicia connecting the target and the area for explanatory text.

14. A method of taking notes of an event on a computerized notetaking device that has a display and at least when needed access to a recording device at least when needed an audio output device, said method comprising the steps of:

(a) said recording device making an audio recording of said event;

(b) said computerized notetaking device inputting notes about said event from a user and timestamping a plurality of these notes to synchronize them with said audio recording;

(c) said computerized notetaking device attempting speech recognition of the audio recording so as to generate automatically recognized text that is also linked to said audio recording, and said computerized notetaking device displaying at least a portion of said automatically recognized text in manner that is distinguishable from said notes; and

(d) said computerized notetaking device inputting a play command that is selective of a portion of said automatically recognized text and responsive to said play command, playing the portion of the audio recording that was translated to said selected portion of said automatically recognized text.

15. The method of claim 14 additionally comprising the step of said computerized notetaking device, having access to both the audio recording and the notes, inputting a timestamp-selective play command from a user and responsive to the play command, playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter;

16. The method of claim 14 additionally comprising the step of said computerized notetaking device inputting a timestamp-selective display command and responsive to said timestamp-selective display command displaying the portion of the automatically recognized text that corresponds to the selected notes.

17. The method of claim 14 wherein during the step of attempting speech recognition, said computerized notetaking device examining said notes and looking for words in said notes that might be correct translates of any portion of the audio recording and whenever any word found in said notes might be a correct translate of a portion of the audio recording, using said word that was found rather than any other potential translates of that portion of the audio recording, thereby increasing the accuracy of the attempted speech recognition.

18. A method of taking notes of an event on a computerized notetaking device that has a display and at least when needed access to a recording device at least when needed an audio output device, said method comprising the steps of:

(a) said recording device making an audio recording of said event;

(b) said computerized notetaking device inputting notes about said event from a user and inserting these notes into a document and displaying at least a portion of said document on said display and timestamping a plurality of these notes to synchronize them with said audio recording;

(c) said computerized notetaking device, having access to both the audio recording and the notes, inputting a timestamp-selective play command from a user and responsive to the play command, playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter; and

(d) said computerized notetaking device playing background audio when not playing said audio recording, and additionally responsive to said play command, interrupting the background audio while playing said audio recording.

19. A method of taking notes of an event on a computerized notetaking device that has a display and at least when needed access to a recording device at least when needed an audio output device, said method comprising the steps of:

(a) said recording device making an audio recording of said event;

(b) said computerized notetaking device inputting notes about said event from a user and inserting these notes into a document and displaying at least a portion of said document on said display and timestamping a plurality of these notes to synchronize them with said audio recording;

(c) said computerized notetaking device, having access to both the audio recording and the notes, inputting a timestamp-selective play command from a user and responsive to the play command, playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter; and

(d) said computerized notetaking device, prior to playing said audio recording starting from a particular point, searching the audio recording for a pause in the sound level of the audio recording that is near said particular point and upon finding such a pause further adjusting said particular point so play starts near the end of said pause that was found.

20. The method of claim 19 additionally comprising the step of said computerized notetaking device, when responding to a lockstep play command that is accompanied by a scheduled stop point, searching the audio recording for a pause in the sound level of the audio recording that is near the scheduled stop point and upon finding such a pause stopping audio playback in the pause that was found;

21. A method of taking notes of an event on a computerized notetaking device that has a display and at least when needed access to a recording device at least when needed an audio output device, said method comprising the steps of:

(a) said recording device making an audio recording of said event;

(b) said computerized notetaking device inputting notes about said event from a user and inserting these notes into a document and displaying at least a portion of said document on said display and timestamping a plurality of these notes to synchronize them with said audio recording;

(c) said computerized notetaking device, having access to both the audio recording and the notes, inputting a timestamp-selective play command from a user and responsive to the play command, playing said audio recording starting from the selected timestamp'"'"'s point of synchronization as adjusted by a preplay parameter; and

(d) said computerized notetaking device, while inputting notes to a first location, also inputting a next-topic command and responsive to said next-topic command preparing a second location for notes and establishing a timestamp link that does not link to the notes at the first location but rather links to the second location for notes, and after responding to said next-topic command continuing to input notes to the first location;

22. The method of claim 21 wherein the step of responding to said next-topic command additionally comprises the steps of prompting for a next-topic label, and after inputting a termination to the next-topic label displaying the next-topic label at the second location and continuing to input notes to the first location.

23. A computerized notetaking device comprising:

(a) notes input means;

(b) display means for displaying notes that were input;

(c) audio recording means for creating an audio recording;

(d) selective audio output means for playing selected portions of the audio recording;

(e) timestamp means for linking a plurality of notes as they are input, each note to a particular point in the audio recording that was recorded about when the note was input, so as to synchronize the notes and the audio recording;

(f) timestamp-selective play command means for inputting a play command that is selective of one of the links between the notes and the audio recording and playing the audio recording starting from a point in the audio recording that was recorded few seconds prior to when the selected link'"'"'s note was input; and

(g) background audio means for playing background audio when not playing any portion of the audio recording and for allowing the playing of the audio recording to interrupt the playing of background audio.

24. A computerized notetaking device comprising:

(a) notes input means;

(b) display means for displaying notes that were input;

(c) audio recording means for creating an audio recording;

(d) selective audio output means for playing selected portions of the audio recording;

(e) timestamp means for linking a plurality of notes as they are input, each note to a particular point in the audio recording that was recorded about when the note was input, so as to synchronize the notes and the audio recording;

(f) timestamp-selective play command means for inputting a play command that is selective of one of the links between the notes and the audio recording and playing the audio recording starting from a point in the audio recording that was recorded few seconds prior to when the selected link'"'"'s note was input;

(g) always-on recording means on said device for always recording audio; and

(h) external timestamp button means accessible from the outside of said device for inputting a timestamp command even when the computerized notetaking device is not in use for taking notes.