System for capturing and retrieving audio data and corresponding hand-written notes

• US 5,970,455 A
• Filed: 03/20/1997
• Issued: 10/19/1999
• Est. Priority Date: 03/20/1997
• Status: Expired due to Term

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.), Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.), Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Sullivan, Joseph W., Sawhney, Nitin, Schilit, William N., Wilcox, Lynn D., Bickmore, Timothy W.
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US08/821,311
• Time in Patent Office: 943 Days
• Field of Search: 704/260, 704/270, 704/275-278, 345/326, 707/200, 707/500, 707/501, 707/530
• US Class Current: 704/270
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

An electronic notebook assigns properties to digital ink of note marks and keywords to digital pages. The properties indicate the type of note mark and the keywords indicate general content of the note marks on a digital page. The notebook also synchronizes audio data with the note marks and permits audio highlighting. The electronic notebook provides an updatable database of note marks and audio that is capable of being searched based on the properties, keywords, and audio highlights.

Claims

1. A digital note-taking system, comprising:

a pen-based interactive display adapted to receive pen-based input;

a processor in communication with the pen-based interactive display and responsive to the pen-based input for generating at least one note mark comprising digital ink for display on the pen-based interactive display, wherein the digital ink comprises at least one user defined property that indicates a category of information expressed by the digital ink; and

a storage device in communication with the processor for storing the digital ink.

2. The digital note-taking system of claim 1, wherein the processor assigns at least one note mark to a digital page, wherein the digital page comprises identifying keywords.

3. The digital note-taking system of claim 1, further comprising:

an audio input device in communication with the processor and responsive to audio for generating audio data, wherein the processor designates at least one highlighted audio data portion.

4. The digital note-taking system of claim 3, wherein the audio data and the digital ink further comprise time data.

5. The digital note-taking system of claim 3, wherein the storage device stores the at least one highlighted audio data portion.

6. The digital note-taking system of claim 3, wherein the processor is adapted to compress at least one non-highlighted audio portion and wherein the storage device stores the compressed at least one non-highlighted audio portion.

7. The digital note-taking system of claim 3, wherein the pen-based interactive display comprises an audio data time line.

8. The digital note-taking system of claim 7, wherein the audio data time line indicates the at least one highlighted audio data portion.

9. The digital note-taking system of claim 3, wherein the at least one note mark corresponds to the at least one highlighted audio data portion.

10. The digital note-taking system of claim 9, wherein the correspondence of the at least one note mark to the at least one highlighted audio data portion is indicated to the user on the display with thicker line width.

11. The digital note-taking system of claim 1, wherein the at least one property is indicated to the user on the display with a color.

12. The digital note-taking system of claim 1, wherein the at least one property is one of the set comprising:

action item, name, calendar item and URL.

13. The digital note-taking system of claim 2, wherein the digital page comprises at least one property specifying region.

14. The digital note-taking system of claim 3, further comprising an audio output device in communication with the processor.

15. The digital note-taking system of claim 14, wherein the processor modifies the at least one highlighted audio data portion for output through the audio output device.

16. The digital note-taking system of claim 3, wherein the processor designates the at least one highlighted audio data portion based upon user input.

17. The digital note-taking system of claim 3, wherein the processor designates the at least one highlighted audio data portion by processing the audio data with a speaker segmentation algorithm.

18. The digital note-taking system of claim 1, wherein the processor is responsive to a user inputted query that identifies a property for searching the storage device and for generating a digital page comprising at least one note mark having digital ink that comprises the identified property.

19. The digital note-taking system of claim 2, wherein the processor is responsive to a user inputted query that identifies a keyword for searching the storage device and for generating a digital page comprising at least one note mark from a digital page that comprises the identified keyword.

20. The digital note-taking system of claim 2, wherein the digital page comprises a background.

21. The digital note-taking system of claim 2, wherein the processor is responsive to a pen-based input comprising a horizontal line gesture to generate a digital page.

22. The digital note-taking system of claim 2, further comprising a browsing window that lists at least one digital page along with the date of creation of the digital page, date of modification of the digital page, at least one icon identifying the property associated with the digital ink on the digital page and at least one keyword associated with the digital page.

23. The digital note-taking system of claim 2, further comprising a query window for generating a query based on at least one of a keyword and property.