Differential dynamic content delivery with text display in dependence upon simultaneous speech

• US 8,332,220 B2
• Filed: 03/25/2008
• Issued: 12/11/2012
• Est. Priority Date: 01/13/2004
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Bodin, William K., Burkhart, Michael J., Eisenhauer, Daniel G., Schumacher, Daniel M., Watson, Thomas J.
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US12/055,240
• Publication Number:
  US 20080172227A1
• Time in Patent Office: 1,722 Days
• Field of Search: 704/203, 704/235, 704/275, 704/200, 704/221, 704/239, 704/243, 704/246, 704/260, 704/270, 709/204, 709/203, 709/231, 707/3, 707/5, 715/202, 715/230, 715/234, 715/700, 715/727
• US Class Current: 704/235
• CPC Class Codes: G01L 19/04 Means for compensating for effects of changes of temperature , i.e. other than electric compensation
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G09B 1/00 Manually or mechanically operated educational appliances using elements forming, or bearing, symbols, signs, pictures, or the like which are arranged or adapted to be arranged in one or more particular ways puzzle-games A63F9/00; advertising or displaying in general G09F
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 19/08 Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 1/00 Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof

Abstract

Differential dynamic content delivery including providing a session document for a presentation, wherein the session document includes a session grammar and a session structured document; selecting from the session structured document a classified structural element in dependence upon user classifications of a user participant in the presentation; presenting the selected structural element to the user; streaming presentation speech to the user including individual speech from at least one user participating in the presentation; converting the presentation speech to text; detecting whether the presentation speech contains simultaneous individual speech from two or more users; and displaying the text if the presentation speech contains simultaneous individual speech from two or more users.

Claims

1. A method comprising:

providing to a multiplicity of users a presentation;

streaming streamed speech to a first user including individual speech from at least one other user participating in the presentation;

detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously; and

using at least one processor to cause a display device to display, to the first user during the presentation, text of the streamed speech in response to detecting that the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously, wherein the two or more users speaking simultaneously comprise a second user and a third user, and wherein the displayed text comprises text corresponding to individual speech from the second user and text corresponding to individual speech from the third user.

2. The method of claim 1, wherein detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously comprises identifying a period of simultaneous speech, and displaying text of the streamed speech comprises displaying text of the streamed speech for a period longer than the period of simultaneous speech.

3. The method of claim 1, wherein detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously comprises detennining whether a period of individual speech from the second user overlaps a period of individual speech from the third user.

4. The method of claim 1 wherein displaying the text further comprises segregating display text.

5. The method of claim 4 wherein segregating display text further comprises segregating display text according to channel, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech transmitted on a first channel; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech transmitted on a second channel.

6. The method of claim 4, wherein segregating display text further comprises segregating display text according to user, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech attributed to the second user; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech attributed to the third user.

7. The method of claim 1 wherein providing to a multiplicity of users a presentation comprises:

providing a session document for the presentation, wherein the session document includes a session grammar and a session structured document;

selecting from the session structured document a classified structural element in dependence upon user classifications of a user participant in the presentation; and

presenting the selected structural element to the user participant.

8. The method of claim 7 wherein selecting a classified structural element further comprises selecting a classified structural element having an associated classification identifier that corresponds to the user classification.

9. The method of claim 7 further comprising creating a session document from a presentation document, including:

identifying a presentation document for a presentation, the presentation document including a presentation grammar and a structured document having structural elements classified with classification identifiers;

identifying a user participant for the presentation, the user participant having a user profile comprising user classifications; and

filtering the structured document in dependence upon the user classifications and the classification identifiers.

10. The method of claim 9 further comprising filtering the presentation grammar, in dependence upon the extracted structural elements, into a session grammar in the session document.

11. The method of claim 9 further comprising creating a presentation document, including:

creating, in dependence upon an original document, a structured document comprising one or more structural elements;

classifying a structural element of the structured document according to a presentation attribute; and

creating a presentation grammar for the structured document, wherein the presentation grammar for the structured document includes grammar elements each of which includes an identifier for at least one structural element of the structured document.

12. The method of claim 11 wherein classifying a structural element comprises:

identifying a presentation attribute for the structural element;

identifying a classification identifier in dependence upon the presentation attribute; and

inserting the classification identifier in association with the structural element in the structured document.

13. The method of claim 11 wherein creating a presentation grammar for the structured document comprises:

identifying the content type of the original document;

selecting, in dependence upon the content type, a full presentation grammar from among a multiplicity of full presentation grammars; and

filtering the full presentation grammar into a presentation grammar for the structured document in dependence upon the structural elements of the structured document.

14. The method of claim 1, wherein:

the first user participates in the presentation from a first computer;

the at least one other user participates in the presentation from a second computer different from the first computer; and

streaming streamed speech to a first user comprises transmitting in real time speech captured at the second computer via at least one network to the first computer.

15. A system comprising at least one processor programmed to:

provide to a multiplicity of users a presentation;

stream streamed speech to a first user including individual speech from at least one other user participating in the presentation;

detect whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously; and

display, to the first user during the presentation, text of the streamed speech in response to detecting that the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously, wherein the two or more users speaking simultaneously comprise a second user and a third user, and wherein the displayed text comprises text corresponding to individual speech from the second user and text corresponding to individual speech from the third user.

16. The system of claim 15 wherein the at least one processor is further programmed to:

identify a period of simultaneous speech resulting from two or more users speaking simultaneously; and

display text of the streamed speech for a period longer than the period of simultaneous speech.

17. The system of claim 15, wherein the at least one processor is further programmed to determine whether a period of individual speech from the second user overlaps a period of individual speech from the third user.

18. The system of claim 15 wherein the at least one processor is further programmed to segregate display text.

19. The system of claim 18 wherein the at least one processor is further programmed to segregate display text according to channel, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech transmitted on a first channel; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech transmitted on a second channel.

20. The system of claim 18, wherein the at least one processor is further programmed to segregate display text according to user, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech attributed to the second user; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech attributed to the third user.

21. The system of claim 15 wherein the at least one processor is further programmed to:

provide a session document for the presentation, wherein the session document includes a session grammar and a session structured document;

select from the session structured document a classified structural element in dependence upon user classifications of a user participant in the presentation; and

present the selected structural element to the user participant.

22. The system of claim 21 wherein the at least one processor is further programmed to select a classified structural element having an associated classification identifier that corresponds to the user classification.

23. The system of claim 21 wherein the at least one processor is further programmed to create a session document from a presentation document, at least in part by:

identifying a presentation document for a presentation, the presentation document including a presentation grammar and a structured document having structural elements classified with classification identifiers;

identifying a user participant for the presentation, the user participant having a user profile comprising user classifications; and

filtering the structured document in dependence upon the user classifications and the classification identifiers.

24. The system of claim 23 wherein the at least one processor is further programmed to filter the presentation grammar, in dependence upon the extracted structural elements, into a session grammar in the session document.

25. The system of claim 23 wherein the at least one processor is further programmed to create a presentation document, at least in part by:

creating, in dependence upon an original document, a structured document comprising one or more structural elements;

classifying a structural element of the structured document according to a presentation attribute; and

creating a presentation grammar for the structured document, wherein the presentation grammar for the structured document includes grammar elements each of which includes an identifier for at least one structural element of the structured document.

26. The system of claim 25 wherein the at least one processor is further programmed to classify a structural element, at least in part by:

identifying a presentation attribute for the structural element;

identifying a classification identifier in dependence upon the presentation attribute; and

inserting the classification identifier in association with the structural element in the structured document.

27. The system of claim 25 wherein the at least one processor is further programmed to create a presentation grammar for the structured document, at least in part by:

identifying the content type of the original document;

selecting, in dependence upon the content type, a full presentation grammar from among a multiplicity of full presentation grammars; and

filtering the full presentation grammar into a presentation grammar for the structured document in dependence upon the structural elements of the structured document.

28. The system of claim 15, wherein:

the first user participates in the presentation from a first computer;

the at least one other user participates in the presentation from a second computer different from the first computer; and

the at least one processor is further programmed to transmit in real time speech captured at the second computer via at least one network to the first computer.

29. At least one computer readable recordable medium encoded with instructions that, when executed, perform a method comprising:

providing to a multiplicity of users a presentation;

streaming streamed speech to a first user including individual speech from at least one other user participating in the presentation;

detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously; and

displaying, to the first user during the presentation, text of the streamed speech in response to detecting that the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously, wherein the two or more users speaking simultaneously comprise a second user and a third user, and wherein the displayed text comprises text corresponding to individual speech from the second user and text corresponding to individual speech from the third user.

30. The at least one computer readable recordable medium of claim 29, wherein detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously comprises identifying a period of simultaneous speech, and displaying text of the streamed speech comprises displaying text of the streamed speech for a period longer than the period of simultaneous speech.

31. The at least one computer readable recordable medium of claim 29, wherein detecting whether the streamed speech contains simultaneous speech resulting from two or more users speaking simultaneously comprises determining whether a period of individual speech from the second user overlaps a period of individual speech from the third user.

32. The at least one computer readable recordable medium of claim 29, wherein displaying the text further comprises segregating display text.

33. The at least one computer readable recordable medium of claim 32, wherein segregating display text further comprises segregating display text according to channel, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech transmitted on a first channel; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech transmitted on a second channel.

34. The at least one computer readable recordable medium of claim 32, wherein segregating display text further comprises segregating display text according to user, comprising:

displaying, in a first display area, a first portion of the text corresponding to a first portion of the streamed speech attributed to the second user; and

displaying, in a second display area, a second portion of the text corresponding to a second portion of the streamed speech attributed to the third user.

35. The at least one computer readable recordable medium of claim 29, wherein providing to a multiplicity of users a presentation comprises:

providing a session document for the presentation, wherein the session document includes a session grammar and a session structured document;

selecting from the session structured document a classified structural element in dependence upon user classifications of a user participant in the presentation; and

presenting the selected structural element to the user participant.

36. The at least one computer readable recordable medium of claim 35, wherein selecting a classified structural element further comprises selecting a classified structural element having an associated classification identifier that corresponds to the user classification.

37. The at least one computer readable recordable medium of claim 35, wherein the method further comprises creating a session document from a presentation document, including:

identifying a presentation document for a presentation, the presentation document including a presentation grammar and a structured document having structural elements classified with classification identifiers;

identifying a user participant for the presentation, the user participant having a user profile comprising user classifications; and

filtering the structured document in dependence upon the user classifications and the classification identifiers.

38. The at least one computer readable recordable medium of claim 37, wherein the method further comprises filtering the presentation grammar, in dependence upon the extracted structural elements, into a session grammar in the session document.

39. The at least one computer readable recordable medium of claim 37, wherein the method further comprises creating a presentation document, including:

creating, in dependence upon an original document, a structured document comprising one or more structural elements;

classifying a structural element of the structured document according to a presentation attribute; and

creating a presentation grammar for the structured document, wherein the presentation grammar for the structured document includes grammar elements each of which includes an identifier for at least one structural element of the structured document.

40. The at least one computer readable recordable medium of claim 39 wherein, classifying a structural element comprises:

identifying a presentation attribute for the structural element;

identifying a classification identifier in dependence upon the presentation attribute; and

inserting the classification identifier in association with the structural element in the structured document.

41. The at least one computer readable recordable medium of claim 39, wherein creating a presentation grammar for the structured document comprises:

identifying the content type of the original document;

selecting, in dependence upon the content type, a full presentation grammar from among a multiplicity of full presentation grammars; and

filtering the full presentation grammar into a presentation grammar for the structured document in dependence upon the structural elements of the structured document.

42. The at least one computer readable recordable medium of claim 35, wherein:

the first user participates in the presentation from a first computer;

the at least one other user participates in the presentation from a second computer different from the first computer; and

streaming streamed speech to a first user comprises transmitting in real time speech captured at the second computer via at least one network to the first computer.