VOICE STREAM AUGMENTED NOTE TAKING

• US 20110288863A1
• Filed: 05/21/2010
• Published: 11/24/2011
• Est. Priority Date: 05/21/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rasmussen, David John
• Granted Patent: US 9,236,047 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Voice stream augmented note taking may be provided. An audio stream associated with at least one speaker may be recorded and converted into text chunks. A text entry may be received from a user, such as in an electronic document. The text entry may be compared to the text chunks to identify matches, and the matching text chunks may be displayed to the user for selection.

Claims

1. A method for providing voice stream augmented note taking, the method comprising:

recording an audio stream associated with at least one speaker;

converting the audio stream into at least one text chunk;

receiving at least one text entry from a user;

determining whether the at least one text entry from the user is associated with the at least one text chunk; and

in response to determining that the at least one text entry from the user is associated with the at least one text chunk, displaying the at least one text chunk to the user as a suggestion.

2. The method of claim 1, wherein converting the audio stream into at least one text chunk comprises:

performing a speech to text conversion on the audio stream; and

identifying at least one text chunk boundary according to at least one of the following;

a phrase boundary, a sentence boundary, and a time boundary.

3. The method of claim 1, wherein the at least one text chunk is associated with a timestamp.

4. The method of claim 3, wherein the received at least one text entry is associated with an electronic document.

5. The method of claim 4, further comprising:

receiving a selection of the displayed at least one text chunk from the user; and

inserting the displayed at least one text chunk into the electronic document.

6. The method of claim 5, further comprising displaying at least one second converted text chunk to the user.

7. The method of claim 6, wherein the at least one second converted text chunk comprises a next subsequent text chunk following the at least one text chunk.

8. The method of claim 1, further comprising:

identifying a plurality of converted text chunks at least partially matching the at least one text entry; and

displaying the plurality of converted text chunks to the user as a selectable list.

9. The method of claim 8, wherein each of the plurality of converted text chunks is associated with a time stamp.

10. The method of claim 9, wherein each of the plurality of converted text chunks are displayed in the selectable list in chronological order.

11. The method of claim 9, wherein each of the plurality of converted text chunks are displayed in the selectable list in reverse chronological order.

12. The method of claim 8, wherein each of the plurality of converted text chunks are displayed in the selectable list in order of relevance to the at least one text entry.

13. A computer-readable medium which stores a set of instructions which when executed performs a method for providing voice stream augmented note taking, the method executed by the set of instructions comprising:

recording a voice stream into a buffer;

converting the voice stream into a text stream;

identifying at least one text chunk associated with the text stream;

receiving a text input to an electronic document from a user;

determining whether the text input at least partially matches the at least one text chunk;

in response to determining that the text input at least partially matches the at least one text chunk, displaying the at least one text chunk to the user as a selectable element;

receiving a selection of the displayed at least one text chunk from the user; and

inserting the at least one text chunk into the electronic document.

14. The computer-readable medium of claim 13, further comprising:

identifying a plurality of text chunks associated with the text stream, wherein each of the plurality of text chunk is identified according to at least one boundary and wherein the at least one boundary comprises at least one of the following;

a pause in the recorded voice stream, a sentence boundary, a conjunction word within the text stream, and a phrase boundary.

15. The computer-readable medium of claim 14, further comprising displaying each of the plurality of text chunks that at least partially match the text input to the user as a list of selectable elements.

16. The computer-readable medium of claim 15, further comprising:

receiving a second text input from the user; and

removing, from the list of selectable elements, any of the displayed plurality of text chunks that do not at least partially match the second text input.

17. The computer-readable medium of claim 15, wherein the list of selectable elements is ordered according to at least one of the following:

a time stamp and a relevance to the text input.

18. The computer-readable medium of claim 13, further comprising:

in response to receiving the selection of the displayed at least one text chunk from the user, displaying at least one subsequent text chunk to the user as a selectable element.

19. The computer-readable medium of claim 13, wherein the buffer is operative to store a configurable time period of the recorded voice stream.

20. A system for providing voice stream augmented note taking, the system comprising:

a memory storage; and

a processing unit coupled to the memory storage, wherein the processing unit is operative to;

record a speech stream into a buffer, wherein the buffer comprises an amount of memory within the memory storage sufficient to store a configurable time period of the recorded speech stream,

convert the recorded speech stream stored in the buffer into a corresponding text stream comprising a plurality of word,

associate metadata comprising a timestamp with each of the plurality of words,

identify at least one text chunk within the corresponding text stream, wherein the at least one text chunk comprises at least one of the following;

a pause, a sentence boundary, a conjunction word, and a phrase boundary,

receive a text input into an electronic document from a user of the system,

determine whether the text input at least partially matches the at least one text chunk, wherein the at least one text chunk comprises a most recently identified text chunk within the corresponding text stream,

in response to determining that the text input at least partially matches the at least one text chunk, display the at least one text chunk as a suggested text chunk to the user of the system,

determine whether the text input at least partially matched at least one second identified text chunk,

in response to determining that the text input at least partially matched at least one second identified text chunk, display the at least one second text chunk as a second suggested text chunk to the user of the system,

receive a selection of at least one of the at least one text chunk and the at least one second text chunk from the user of the system,

insert the selected at least one of the at least one text chunk and the at least one second text chunk into the electronic document, and

display at least one third text chunk as a third suggested text chunk, wherein the at least one third text chunk comprises a sequentially later text chunk from the selected at least one of the at least one text chunk and the at least one second text chunk.