Aural skimming and scrolling

• US 20080086303A1
• Filed: 11/15/2006
• Published: 04/10/2008
• Est. Priority Date: 09/15/2006
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Sengamedu, Srinivasan H.
• Granted Patent: US 9,087,507 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination

Abstract

Computer-based skimming and scrolling of aurally presented information is described. Different levels of skimming are achieved in aural presentations with allowing a user to navigate an aural presentation according to significant points identified within an information source. The significant points are identified using various indicia that suggest logical arrangements for the information contained within the source, such as semantics, syntax, typography, formatting, named entities, and markup tags. The identified significant points signal changes in playback mode for the audio presentation, such as different tones, pitches, volumes, or voices. Similar indicia may be used to generate identifying markers from the information source that can be aurally presented in lieu of the information source itself to allow for aural scrolling of the information.

Claims

1. A computer-implemented method for skimming aurally presented information comprising the steps of:

analyzing at least one of a set of characteristics of an information source to identify a set of significant points within the information source;

storing location data that identifies the location of the significant points;

receiving an input, and

in response to the input;

inspecting the location data to identify a particular significant point within the information source; and

initiating an aural presentation of the information source at the location of the particular significant point.

2. The computer-implemented method as recited in claim 1 wherein the set of significant points comprises a first set of points and at least a second set of points wherein the method further comprises storing metadata that indicates that the first set of points have a first logical significance and that the second set of points have a second logical significance.

3. The computer-implemented method as recited in claim 2 wherein the particular significant point comprises a member of the first set of points.

4. The computer-implemented method as recited in claim 1 wherein:

the location data is stored in a structure comprising at least one of a hierarchical structure and another internal structure representation, wherein the structure comprises nodes representing segments of the information source; and

the aural presentation of the information source is made according to a sequence based upon the at least one hierarchical and other internal representation structure and the user input.

5. The computer-implemented method as recited in claim 4 wherein the sequence reflects an ordering of segments according to a perceived significance for each segment.

6. The computer-implemented method as recited in claim 1 wherein the input comprises at least one of an aural input and one or more speech inputs.

7. The computer-implemented method as recited in claim 1 wherein the input comprises a tactile input.

8. The computer-implemented method as recited in claim 7 wherein the tactile input is received from an interface comprising at least one of a keyboard, mouse, a button, a joystick, a touchpad, a sensor bearing glove, and a speech input interface.

9. The computer-implemented method as recited in claim 1 wherein the input is received while the information source is being aurally presented from a current playback point.

10. The computer-implemented method as recited in claim 9 wherein the particular significant point precedes the current playback point within the information source.

11. The computer-implemented method as recited in claim 9 wherein the particular significant point follows the current playback point within the information source.

12. The computer-implemented method as recited in claim 1 wherein the information source comprises text-based information.

13. The computer-implemented method as recited in claim 1 wherein the information source comprises at least one of:

an electronic mail message;

output of a messaging client;

a voicemail message;

a document produced by an optical content recognition application;

an electronic document;

textual output of a software application;

an audio stream with accompanying transcription; and

a video stream with accompanying transcription.

14. The computer-implemented method as recited in claim 1 wherein, prior to the analyzing step, the information source is converted into representative text.

15. The computer-implemented method as recited in claim 1 wherein the particular significant point is identified as a significant point based on at least one of:

a font characteristic that changes near the location associated with the particular significant point;

a typographic characteristic that changes near the location associated with the particular significant point;

a semantic significance identified near the location associated with the particular significant point;

a syntactic significance identified near the location associated with the particular significant point;

a named entity identified near the location associated with the particular significant point;

prosodic information associated with the location associated with the particular significant point; and

a markup tag identified near the location associated with the particular significant point.

18. The computer-implemented method as recited in claim 15 wherein the aural presentation in the second playback mode comprises skipping to a significant point other than the particular significant point.

19. The computer-implemented method as recited in claim 15 wherein the information source comprises text-based information.

20. The computer-implemented method as recited in claim 15 wherein the information source comprises at least one of:

an electronic mail message;

output of a messaging client;

a voicemail message;

a document produced by an optical content recognition application;

an electronic document;

textual output of a software application;

an audio stream with accompanying transcription; and

a video stream with accompanying transcription.

21. The computer-implemented method as recited in claim 15 wherein, prior to the analyzing step, the information source is converted into representative text.

22. The computer-implemented method as recited in claim 15 wherein the particular significant point is identified as a significant point based on at least one of:

a font characteristic that changes near the location associated with the particular significant point;

a typographic characteristic that changes near the location associated with the particular significant point;

a semantic significance identified near the location associated with the particular significant point;

a syntactic significance identified near the location associated with the particular significant point;

a named entity identified near the location associated with the particular significant point;

prosodic information associated with the location associated with the particular significant point; and

a markup tag identified near the location associated with the particular significant point.

39. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

40. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

41. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

42. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

43. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

44. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

45. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

46. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

47. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

48. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

49. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

50. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

51. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.

52. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 14.

53. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 15.

56. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 18.

57. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 19.

58. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 20.

59. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 21.

60. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 22.

16. A computer-implemented method for aurally presenting information, comprising:

analyzing at least one of a set of characteristics of an information source to identify a set of significant points within the information source;

storing location data that identifies a location of the significant points;

during an aural presentation of the information source in a first playback mode, using the location data to determine that a current playback location matches a particular significant point; and

in response to detecting that the current playback location matches the particular significant point, changing from the first playback mode to at least a second playback mode.

17. The computer-implemented method as recited in claim 16 wherein the aural presentation in at least the second playback mode differs from the aural presentation in the first playback mode in at least one of a set of characteristics comprising one or more of tone, timbre, pitch, speed, voice, and accent;

wherein the second playback mode comprises one of a plurality of playback modes.

54. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 16.

55. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 17.

23. A computer-implemented method for scrolling aurally presented information comprising:

analyzing at least one of a set of characteristics of an information source to generate a set of identifying markers associated with locations within the information source;

storing location data that identifies locations, within the information source, associated with the identifying markers;

while aurally presenting a particular identifying marker, receiving input; and

in response to the input;

inspecting the location data to identify a location associated with the particular identifying marker; and

initiating an aural presentation of the information source at the location.

24. The computer-implemented method as recited in claim 23 further comprising:

aurally presenting a plurality of the identifying markers in a sequence; and

wherein the step of receiving input occurs while the particular identifying marker is being presented in the sequence;

25. The computer-implemented method as recited in claim 24 wherein the sequence corresponds to at least one of the chronological order and the sequential order of the associated locations within the information source.

26. The computer-implemented method as recited in claim 24 further comprising:

aurally presenting at least a portion of the information source;

wherein the sequence begins with an identifying marker associated with the location of a current playback point in the aural presentation.

27. The computer-implemented method as recited in claim 24 wherein the sequence corresponds to an order associated with the set of identifying markers.

28. The computer-implemented method as recited in claim 24 wherein the sequence reflects a perceived significance for each identifying marker.

29. The computer-implemented method as recited in claim 24 wherein the set of identifying markers comprises a first set of identifying markers and at least a second set of identifying markers, the method further comprising storing metadata that indicates that the first set of identifying markers have a first logical significance and that the at least second set of identifying markers have at least a second logical significance.

30. The computer-implemented method as recited in claim 29 wherein the plurality of identifying markers comprise identifying markers belonging to the first set of identifying markers.

31. The computer-implemented method as recited in claim 23 wherein the input comprises at least one of an aural input and a text based input.

32. The computer-implemented method as recited in claim 23 wherein the input comprises at least one of a speech based input and a tactile input.

33. The computer-implemented method as recited in claim 32 wherein the tactile input is received from an interface comprising at least one of a keyboard, a mouse, a joystick, a touchpad, a sensor bearing glove, a speech input interface, and a button.

34. The computer-implemented method as recited in claim 23 wherein the information source comprises a text-based information source.

35. The computer-implemented method as recited in claim 23 wherein the information source comprises at least one of:

an electronic mail message;

output of a messaging client;

a voicemail message;

a document produced by an optical content recognition application;

an electronic document;

textual output of a software application;

an audio stream with accompanying transcription; and

a video stream with accompanying transcription.

36. The computer-implemented method as recited in claim 23 wherein, prior to the analyzing step, the information source is converted into representative text.

37. The computer-implemented method as recited in claim 23 wherein the particular identifying marker comprises an excerpt of the information source identified based on at least one of:

a font characteristic that changes near the location associated with the particular identifying marker;

a typographic characteristic that changes near the location associated with the particular identifying marker;

a semantic significance identified near the location associated with the particular identifying marker;

a syntactic significance identified near the location associated with the particular identifying marker;

a named entity identified near the location associated with the particular identifying marker; and

a markup tag identified near the location associated with the particular identifying marker.

38. The computer-implemented method as recited in claim 23 wherein the particular identifying marker is generated from an analysis of a segment of the information source at the location associated with the particular identifying marker, wherein the analysis comprises at least one of summarization, categorization, shallow parsing, grammar tagging, semantic tagging, and named entity recognition.

61. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 23.

62. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 24.

63. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 25.

64. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 26.

65. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 27.

66. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 28.

67. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 29.

68. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 30.

69. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 31.

70. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 32.

71. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 33.

72. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 34.

73. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 35.

74. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 36.

75. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 37.

76. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 38.