Method and apparatus for generating visual representations for audio documents

• US 6,624,826 B1
• Filed: 09/28/1999
• Issued: 09/23/2003
• Est. Priority Date: 09/28/1999
• Status: Expired due to Term

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Balabanovic, Marko
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): Vu, Kieu D.
• Application Number:
  US09/407,010
• Time in Patent Office: 1,456 Days
• Field of Search: 345/716, 345/727, 345/728, 345/730, 345/760, 345/764, 345/768, 707/512, 704/276, 715/512
• US Class Current: 715/727
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A method and apparatus for generating visual representations for audio documents. A dynamically adjustable audio gauge is used to visually represent an audio narration that may optionally describe an electronic document or image displayed upon a display device. By indicating a position on a display screen through clicking, pointing, or touching the display screen, audio recording is initiated and a dynamically adjustable audio gauge is displayed. The audio gauge increases in size in proportion to the amount of audio recorded while the audio gauge is active. Audio recording may cease upon the audio level dropping below a predetermined threshold or may cease in response to specific user input. For each additional positional stimulus received, a new audio gauge is generated and the previous audio gauge ceases to be adjusted, thereby becoming inactive. The audio gauges may include one or more references to various forms of electronic documents and may be used in an electronic messaging scenario.

Claims

1. A method for generating multimedia representation on a display device comprising:

displaying a representation of an electronic document having a first location upon the display device;

receiving an audio input stream;

displaying a first audio gauge with a graphical connection to the electronic document location upon the display device;

dynamically adjusting a length of the first audio gauge to be proportional in size to an amount of audio recorded;

receiving a second positional stimulus indicating a second location on the display device, and

generating a reference marker connecting an end-point of the first audio gauge to the second location indicated by the second positional stimulus, wherein the reference marker is generated only if the second location on the display device is occupied by an electronic document.

2. The method of claim 1, further comprising:

displaying a second audio gauge located at a second location the second audio gate having a graphical connection to either the first audio gate or a second electronic document; and

dynamically adjusting the second audio gauge to be proportional in size to an amount of audio recorded after the second audio gauge is displayed.

3. The method of claim 2, wherein when the second audio gauge is displayed, the first audio gauge terminates being dynamically updated.

4. The method of claim 2, wherein the graphical connection indicates a chronological order of generation.

5. The method of claim 1, wherein receiving of the audio stream is initiated upon receiving a first positional stimulus corresponding to the first location upon the display device.

6. The method of claim 1, wherein recording of the audio stream is initiated upon detecting audio input above a predetermined threshold level.

7. The method of claim 1, wherein the first audio gauge is rendered semi-transparently so as to allow documents displayed underneath the first audio gauge to remain visible through the first audio gauge.

8. The method of claim 1, wherein the first audio gauge is substantially linear in shape.

9. The method of claim 8, wherein the first audio gauge includes start and stop indicators.

10. The method of claim 1, wherein the reference marker is semi-transparently rendered.

11. The method of claim 1, wherein an image associated with the source of the audio input stream is displayed.

12. The method of claim 11, wherein the image is accessed from a database.

13. The method of claim 11, wherein the image is captured by a video capture device.

14. An article of manufacture comprising a machine readable medium having instructions stored thereon, wherein when the instructions are executed by a processor, the instructions cause the processor to:

cause a representation of an electronic document to be displayed at a first location upon a display device;

receive a first audio input stream;

display a first audio gauge with a graphical connection to the electronic document upon the display device; and

dynamically adjust a length of the first audio gauge to be proportional in size to an amount of audio recorded;

receive a second positional stimulus indicating a second location on the display device, and

generate a reference marker connecting an end-point of the first audio gauge to the second location indicated by the second positional stimulus, wherein the reference marker is generated only if the second location on the display device is occupied by an electronic document.

15. The method of claim 14, further comprising:

receiving at least one additional positional stimulus indicating additional locations upon the display device;

terminating recording of the first audio stream and initiating recording of at least one additional audio stream upon receiving the at least one additional positional stimulus;

displaying at least one additional audio gauge upon receiving the at least one additional positional stimulus; and

dynamically adjusting the at least one additional audio gauge to indicate an amount of audio recorded after receiving the at least one additional positional stimulus.

16. The method of claim 15, wherein recording of each of the at least one additional audio streams is terminated upon receiving a subsequent positional stimulus.

17. The method of claim 15, wherein recording of each of the at least one additional audio streams is terminated upon receiving an amount of audio that is below a predetermined threshold level.

18. A method for generating a multimedia representation comprising:

receiving a first positional stimulus indicating a first location upon a display device;

initiating recording of an audio input stream;

displaying a first audio gauge at the first location upon the display device;

receiving a second positional stimulus indicating a second location upon the display device;

generating a reference marker graphically connecting the audio gauge to the second location;

representing the audio input stream by a first address pointing to a storage location of the audio input stream;

representing the first audio gauge by a second address pointing to a storage location of the first audio gauge;

generating a first graphical representation of the first and second addresses;

representing the first graphical representation by a third address pointing to a storage location of the first graphical representation; and

electronically transferring the third address.

19. The method of claim 18, wherein the second location upon the display device corresponds to an electronic document displayed upon the display device.

20. The method of claim 19, wherein the multimedia representation may be generated in a synchronized multimedia integration language (SMIL) so as to allow playback as a slideshow.

21. The method of claim 18, wherein at least one of the first and second addresses are uniform resource locator (URL) addresses.

22. The method of claim 18, wherein the first graphical representation is a hypertext markup language (HTML) representation.

23. The method of claim 18, wherein the first graphical representation is an extensible markup language (XML) representation.

24. An apparatus for generating multimedia representation on a display device comprising:

means for displaying a representation of an electronic document having a first location upon the display device;

means for receiving an audio input stream;

means for displaying a first audio gauge with a graphical connection to the electronic document location upon the display device;

means for dynamically adjusting a length of the first audio gauge to be proportional in size to an amount of audio recorded;

means for receiving a second positional stimulus indicating a second location on the display device, and

means for generating a reference marker connecting an end-point of the first audio gauge to the second location indicated by the second positional stimulus, wherein the reference marker is generated only if the second location on the display device is occupied by an electronic document.

25. The apparatus of claim 24, wherein the positional stimulus is received from at least one of a touch-screen, mouse, trackball, stylus, and pen.

26. The apparatus of claim 24, further comprising:

means for displaying a second audio gauge located at a second location upon the display device, if a second positional stimulus indicating the second location is received; and

means for dynamically adjusting the second audio gauge to be proportional in size to an amount of audio recorded after the second audio gauge is displayed.

27. The apparatus of claim 26, wherein when the second audio gauge is displayed, the first audio gauge terminates being dynamically updated.

28. The apparatus of claim 26, wherein the first and the second audio gauges are graphically connected together to show a chronological order of generation.

29. The apparatus of claim 24, wherein the first audio gauge is rendered semi-transparently so as to allow documents displayed underneath the first audio gauge to remain visible through the first audio gauge.