Voice stream augmented note taking

• US 9,236,047 B2
• Filed: 05/21/2010
• Issued: 01/12/2016
• Est. Priority Date: 05/21/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Rasmussen, David John
• Primary Examiner(s): PULLIAS, JESSE SCOTT
• Application Number:
  US12/785,353
• Publication Number:
  US 20110288863A1
• Time in Patent Office: 2,062 Days
• Field of Search: 704/231, 704/235, 704/251, 704/270, 704/275
• US Class Current: 1/1
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Voice stream augmented note taking may be provided. An audio stream associated with at least one speaker may be recorded and converted into text chunks. A text entry may be received from a user, such as in an electronic document. The text entry may be compared to the text chunks to identify matches, and the matching text chunks may be displayed to the user for selection.

Claims

1. A method for providing voice stream augmented note taking, the method comprising:

recording an audio stream associated with at least one speaker;

converting the audio stream into text, wherein the text includes at least one text chunk comprising a sentence or phrase, the at least one text chunk comprising content-laden text that is defined by logical breaks identifying phrase or sentence boundaries in the text;

receiving at least one text entry from a user;

determining whether the at least one text entry received from the user matches at least a portion of the at least one text chunk; and

in response to determining that the at least one text entry from the user is associated with the at least one text chunk, displaying the at least one text chunk to the user as a suggestion.

2. The method of claim 1, wherein converting the audio stream into at least one text chunk comprises:

performing a speech to text conversion on the audio stream; and

identifying at least one text chunk boundary according to at least one of the following;

a phrase boundary, a sentence boundary, and a time boundary.

3. The method of claim 1, wherein the at least one text chunk is associated with a timestamp.

4. The method of claim 3, wherein the received at least one text entry is associated with an electronic document.

5. The method of claim 4, further comprising:

receiving a selection of the displayed at least one text chunk from the user; and

inserting the displayed at least one text chunk into the electronic document.

6. The method of claim 5, further comprising displaying at least one second converted text chunk to the user.

7. The method of claim 6, wherein the at least one second converted text chunk comprises a next subsequent text chunk following the at least one text chunk.

8. The method of claim 1, further comprising:

identifying a plurality of converted text chunks at least partially matching the at least one text entry; and

displaying the plurality of converted text chunks to the user as a selectable list.

9. The method of claim 8, wherein each of the plurality of converted text chunks is associated with a time stamp.

10. The method of claim 9, wherein each of the plurality of converted text chunks are displayed in the selectable list in chronological order.

11. The method of claim 9, wherein each of the plurality of converted text chunks are displayed in the selectable list in reverse chronological order.

12. The method of claim 8, wherein each of the plurality of converted text chunks are displayed in the selectable list in order of relevance to the at least one text entry.

13. The method of claim 1, wherein displaying the at least one text chunk to the user as a suggestion includes the at least one text chunk identified in converting the audio stream into the at least one text chunk.

14. The method of claim 1, wherein the phrase or sentence boundaries in the text stream comprise logical breaks in the text stream.

15. A computer-readable storage device which stores a set of instructions which when executed performs a method for providing voice stream augmented note taking, the method executed by the set of instructions comprising:

recording a voice stream into a buffer;

converting the voice stream into a text stream, wherein the text stream includes at least one text chunk comprising a sentence or phrase, the at least one text chunk comprising content-laden text that is defined by logical breaks identifying phrase or sentence boundaries in the text;

receiving a text input to an electronic document from a user;

determining whether the text input at least partially matches the at least one text chunk;

in response to determining that the text input at least partially matches the at least one text chunk, displaying the at least one text chunk to the user as a selectable element;

receiving a selection of the displayed at least one text chunk from the user; and

inserting the at least one text chunk into the electronic document.

16. The computer-readable storage device of claim 15, further comprising:

identifying a plurality of text chunks associated with the text stream, wherein each of the plurality of text chunk is identified according to at least one boundary and wherein the at least one boundary comprises at least one of the following;

a pause in the recorded voice stream, a sentence boundary, a conjunction word within the text stream, and a phrase boundary.

17. The computer-readable storage device of claim 16, further comprising displaying each of the plurality of text chunks that at least partially match the text input to the user as a list of selectable elements.

18. The computer-readable storage device of claim 17, further comprising:

receiving a second text input from the user; and

removing, from the list of selectable elements, any of the displayed plurality of text chunks that do not at least partially match the second text input.

19. The computer-readable storage device of claim 15, further comprising:

in response to receiving the selection of the displayed at least one text chunk from the user, displaying at least one subsequent text chunk to the user as a selectable element.

20. A system for providing voice stream augmented note taking, the system comprising:

a memory storage; and

a processing unit coupled to the memory storage, wherein the processing unit is operative to;

record a speech stream into a buffer, wherein the buffer comprises an amount of memory within the memory storage sufficient to store a configurable time period of the recorded speech stream,

convert the recorded speech stream stored in the buffer into a corresponding text stream comprising a plurality of words,

associate metadata comprising a timestamp with each of the plurality of words,

identify at least one text chunk within the corresponding text stream, wherein the at least one text chunk comprises at least one of the following;

a pause, a sentence boundary, a conjunction word, and a phrase boundary,

receive a text input into an electronic document from a user of the system,

determine whether the text input at least partially matches the at least one text chunk, wherein the at least one text chunk comprises a most recently identified text chunk within the corresponding text stream,

in response to determining that the text input at least partially matches the at least one text chunk, display the at least one text chunk as a suggested text chunk to the user of the system,

determine whether the text input at least partially matched at least one second identified text chunk,

in response to determining that the text input at least partially matched at least one second identified text chunk, display the at least one second text chunk as a second suggested text chunk to the user of the system,

receive a selection of at least one of the at least one text chunk and the at least one second text chunk from the user of the system,

insert the selected at least one of the at least one text chunk and the at least one second text chunk into the electronic document, and

display at least one third text chunk as a third suggested text chunk, wherein the at least one third text chunk comprises a sequentially later text chunk from the selected at least one of the at least one text chunk and the at least one second text chunk.