Systems and methods for freeform annotations

• US 20050183005A1
• Filed: 02/12/2004
• Published: 08/18/2005
• Est. Priority Date: 02/12/2004
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Denoue, Laurent, Golovchinsky, Gene
• Granted Patent: US 7,343,552 B2
• Time in Patent Office: Days
• US Class Current: 715/202
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

Systems and methods in accordance with the present invention provide freeform annotation techniques designed to make freeform annotations on a digital document more flexible, reliable and presentable on a screen of a computing device. They allow users to markup the document, add comments, and link between annotations, while maintaining the freeform nature of these annotations without constraining the kinds of annotations allowed. These freeform annotations are then unequivocally recognized via a state machine into three groups of annotation marks: anchor, comment, and link, each of which is associated with either part of the digital document or with another annotation mark. In addition, these annotation marks can be repositioned and resized to fit within the display area available on the screen based on the display properties of the screen, thus supporting the natural flow of annotating the document in a freeform manner. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

Claims

1. A method to support freeform annotations, comprising:

reading and displaying a portion of a digital document on a screen based on one or more display properties of the screen;

accepting one or more freeform annotations on the displayed portion of the digital document via one or more input devices; and

processing the one or more freeform annotations via a completely deterministic state machine.

2. The method according to claim 1, wherein:

the method can be performed on one or more computing devices, wherein a computing device in the one or more computing devices can be one of;

a PDA, a Tablet PC, a Pocket PC, a cell phone, an electronic messaging device, a Java-enabled device, a laptop or desktop PC, a workstation, and a mainframe computer.

3. The method according to claim 1, wherein:

the method can be implemented in one or more of the following programming languages;

C, C++, Visual Basic, Java and Java Applet.

4. The method according to claim 1, wherein:

the digital document can include at least one of;

a text file, an image, a figure, a drawing, a graph, a picture, and a video clip.

5. The method according to claim 1, wherein:

a display property in the one or more display properties of the screen can be associated with one of;

a text font, a font size, a text color, a display width, height, and resolution of the screen.

6. The method according to claim 1, wherein:

a freeform annotation in the one or more freeform annotations can be one or more of;

strokes, gestures and handwritings of any shape.

7. The method according to claim 1, wherein:

an input device in the one or more input devices can be one of;

a keyboard, a mouse, and a stylus.

8. The method according to claim 1, further comprising:

storing the digital document in memory and/or in a persistent storage.

9. The method according to claim 1, wherein:

the reading and displaying step comprises at least one of;

utilizing the control of the Microsoft Internet Explorer; and

accessing the Document Object Model to obtain the bounding box of the digital document.

10. The method according to claim 1, further comprising:

reserving an area on the screen for the input of the one or more freeform annotations.

11. The method according to claim 1, further comprising:

managing one or more annotation marks on the displayed portion of the digital document by performing at least one of;

creating, selecting, modifying, moving, deleting, rendering, and displaying an annotation mark of the one or more annotation marks on the screen based on the one or more display properties of the screen.

12. The method according to claim 11, wherein:

an annotation mark in the one or more annotation marks can be one of;

an anchor capable of associating with a part of the portion of the digital document;

a link capable of connecting at least two anchors; and

a comment capable of associating with either an anchor or a connector.

13. The method according to claim 12, wherein:

the anchor can be one of;

an underline, a highlight, a circled word, a circled area, a bracket, and a margin bar; and

the link can be one of;

an arrow, and a connector.

14. The method according to claim 11, further comprising:

storing the one or more annotation marks in memory and/or in a persistent storage.

15. The method according to claim 11, wherein:

the selecting, moving, modifying, and deleting step comprises;

tapping, holding, and lifting an input device in the one or more input devices on the annotation mark.

16. The method according to claim 11, wherein:

the rendering step comprises at least one of;

retrieving a bounding box of the annotation mark;

resizing and/or splitting the annotation mark; and

repositioning the annotation mark on the portion of the digital document under display.

17. The method according to claim 12, further comprising:

reserving an area on the screen for the rendering of the comment; and

/or

resizing and/or repositioning the comment based on one of the one or more display properties.

18. The method according to claim 12, further comprising:

identifying the start of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation is capable of;

starting on a blank area on the digital document under display; and

/or

not intersecting with any word or image on the digital document.

19. The method according to claim 12, further comprising:

identifying the finish of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation can be one of;

a click on a predefined area on the screen, which can be a button;

a circular gesture around the comment;

a strike-through gesture on the comment;

a stroke bouncing off from the bottom of the screen;

a special gesture that is rare in common handwriting; and

a vertical stroke drawn on the left or right of the comment.

20. A system to support freeform annotations, comprising:

a storage component capable of maintaining a digital document and one or more software components;

a display component capable of;

displaying a portion of the digital document based on one or more display properties of the display component; and

accepting one or more freeform annotations on the displayed portion of the digital document via one or more input devices; and

a processing component capable of processing the one or more freeform annotations via a completely deterministic state machine.

21. The system according to claim 20, wherein:

the system can be part of one or more computing devices, wherein a computing device in the one or more computing devices can be one of;

a PDA, a Tablet PC, a Pocket PC, a cell phone, an electronic messaging device, a Java-enabled device, a laptop or desktop PC, a workstation, and a mainframe computer.

22. The system according to claim 20, wherein:

the storage component can be a memory or a persistent storage.

23. The system according to claim 20, wherein:

the display component can be a screen, which can be a LCD or a CRT monitor.

24. The system according to claim 20, wherein:

the processing component can be a CPU, an embedded CPU, or a MCU.

25. The system according to claim 20, wherein:

the digital document can include at least one of;

a text file, an image, a figure, a drawing, a graph, a picture, and a video clip.

26. The system according to claim 20, wherein:

a software component in the one or more software components can be implemented in one or more of the following programming languages;

C, C++, Visual Basic, Java and Java Applet.

27. The system according to claim 20, wherein:

a display property in the one or more display properties of the display component can be associated with one of;

a text font, a font size, a text color, a display width, height, and resolution of the display component.

28. The system according to claim 20, wherein:

an input device in the one or more input devices can be one of;

a keyboard, a mouse, and a stylus.

29. The system according to claim 20, wherein:

a freeform annotation in the one or more freeform annotations can be one or more of;

strokes, gestures and handwritings of any shape.

30. The system according to claim 20, wherein:

the display component is further capable of;

utilizing the control of the Microsoft Internet Explorer; and

accessing the Document Object Model to obtain a bounding box of the digital document.

31. The system according to claim 20, wherein:

the display component is further capable of reserving an area on the display component for the input of the one or more freeform annotations.

32. The system according to claim 20, wherein:

the processing component is further capable of;

managing one or more annotation marks on the displayed portion of the digital document by performing at least one of;

creating, selecting, modifying, moving, deleting, rendering, and displaying an annotation mark of the one or more annotation marks on the display component based on the one or more display properties of the display component.

33. The system according to claim 32, wherein:

an annotation mark in the one or more annotation marks can be one of;

an anchor capable of associating with a part of the portion of the digital document;

a link capable of connecting at least two anchors; and

a comment capable of associating with either an anchor or a connector.

34. The system according to claim 33, wherein:

the anchor can be one of;

an underline, a highlight, a circled word, a circled area, a bracket, and a margin bar; and

the link can be one of;

an arrow, and a connector.

35. The system according to claim 32, wherein:

the storage component is further capable of maintaining the one or more annotation marks.

36. The system according to claim 32, wherein:

the processing component is further capable of performing at least one of;

selecting, moving, modifying, and deleting the annotation mark, via tapping, holding, and lifting an input device in the one or more input devices on the annotation mark.

37. The system according to claim 32, wherein:

the processing component is further capable of rendering the annotation mark via at least one of;

retrieving a bounding box of the annotation mark;

resizing and/or splitting the annotation mark; and

repositioning the annotation mark on the portion of the digital document under display.

38. The system according to claim 33, wherein:

the processing component is further capable of rendering the comment by;

reserving an area on the screen for the rendering of the comment; and

/or

resizing and/or repositioning the comment based on one of the one or more display properties.

39. The system according to claim 33, wherein:

the processing component is further capable of identifying the start of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation is capable of;

starting on a blank area on the digital document under display; and

/or

not intersecting with any word or image on the digital document.

40. The system according to claim 33, wherein:

the processing component is further capable of identifying the finish of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation can be one of;

a click on a predefined area on the screen, which can be a button;

a circular gesture around the comment;

a strike-through gesture on the comment;

a stroke bouncing off from the bottom of the screen;

a special gesture that is rare in common handwriting; and

a vertical stroke drawn on the left or right of the comment.

41. A machine readable medium having instructions stored thereon that when executed by a processor cause a system to:

read and display a portion of a digital document on a screen based on one or more display properties of the screen;

accept one or more freeform annotations on the displayed portion of the digital document via one or more input devices; and

process the one or more freeform annotations via a completely deterministic state machine.

42. The machine readable medium of claim 41, wherein:

the instructions can be embedded and executed on one or more computing devices, wherein a computing device in the one or more computing devices can be one of;

a PDA, a Tablet PC, a Pocket PC, a cell phone, an electronic messaging device, a Java-enabled device, a laptop or desktop PC, a workstation, and a mainframe computer.

43. The machine readable medium of claim 41, wherein:

the instructions can be implemented in one or more of the following programming languages;

C, C++, Visual Basic, Java and Java Applet.

44. The machine readable medium of claim 41, wherein:

the digital document can include at least one of;

a text file, an image, a figure, a drawing, a graph, a picture, and a video clip.

45. The machine readable medium of claim 41, wherein:

a display property in the one or more display properties of the screen can be associated with one of;

a text font, a font size, a text color, a display width, height, and resolution of the screen.

46. The machine readable medium of claim 41, wherein:

a freeform annotation in the one or more freeform annotations can be one or more of;

strokes, gestures and handwritings of any shape.

47. The machine readable medium of claim 41, wherein:

an input device in the one or more input devices can be one of;

a keyboard, a mouse, and a stylus.

48. The machine readable medium of claim 41, further comprising instructions that when executed cause the system to:

store the digital document in memory and/or in a persistent storage.

49. The machine readable medium of claim 41, further comprising instructions that when executed cause the system to:

utilize the control of the Microsoft Internet Explorer to read the digital document; and

access the Document Object Model to obtain the one or more display properties.

50. The machine readable medium of claim 41, further comprising instructions that when executed cause the system to:

reserve an area on the screen for the input of the one or more freeform annotations.

51. The machine readable medium of claim 41, further comprising instructions that when executed cause the system to:

manage one or more annotation marks on the displayed portion of the digital document by performing at least one of;

creating, selecting, modifying, moving, deleting, rendering, and displaying an annotation mark of the one or more annotation marks on the screen based on the one or more display properties of the screen.

52. The machine readable medium of claim 51, wherein:

an annotation mark in the one or more annotation marks can be one of;

an anchor capable of associating with the portion of the digital document;

a link capable of connecting at least two anchors; and

a comment capable of associating with either an anchor or a connector.

53. The machine readable medium of claim 52, wherein:

the anchor can be one of;

an underline, a highlight, a circled word, a circled area, a bracket, and a margin bar; and

the link can be one of;

an arrow, and a connector.

54. The machine readable medium of claim 51, further comprising instructions that when executed cause the system to:

store the one or more annotation marks in memory and/or in a persistent storage.

55. The machine readable medium of claim 51, further comprising instructions that when executed cause the system to:

perform at least one of;

selecting, moving, modifying, and deleting the annotation mark via tapping, holding, and lifting an input device in the one or more input devices on the annotation mark.

56. The machine readable medium of claim 51, further comprising instructions that when executed cause the system to:

render the annotation mark via at least one of;

retrieving a bounding box of the annotation mark;

resizing and/or splitting the annotation mark; and

repositioning the annotation mark on the portion of the digital document under display.

57. The machine readable medium of claim 52, further comprising instructions that when executed cause the system to:

reserve an area on the screen for the rendering of the comment; and

/or

resize and/or reposition the comment based on one of the one or more display properties.

58. The machine readable medium of claim 52, further comprising instructions that when executed cause the system to:

identify the start of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation is capable of;

starting on a blank area on the digital document under display; and

/or

not intersecting with any word or image on the digital document.

59. The machine readable medium of claim 52, further comprising instructions that when executed cause the system to:

identify the finish of the comment by recognizing a freeform annotation in the one or more freeform annotations, wherein the freeform annotation can be one of;

a click on a predefined area on the screen, which can be a button;

a circular gesture around the comment;

a strike-through gesture on the comment;

a stroke bouncing off from the bottom of the screen;

a special gesture that is rare in common handwriting; and

a vertical stroke drawn on the left or right of the comment.

60. A system for managing versions of a software component, comprising:

means for reading and displaying a portion of a digital document on a screen based on one or more display properties of the screen;

means for accepting one or more freeform annotations on the displayed portion of the digital document via one or more input devices; and

means for processing the one or more freeform annotations via a completely deterministic state machine.

61. A computer data signal embodied in a transmission medium, comprising:

a code segment including instructions to read and display a portion of a digital document on a screen based on one or more display properties of the screen;

a code segment including instructions to accept one or more freeform annotations on the displayed portion of the digital document via one or more input devices; and

a code segment including instructions to process the one or more freeform annotations via a completely deterministic state machine.