Method and apparatus for creating and editing node-link diagrams in pen computing systems

• US 8,014,607 B2
• Filed: 03/23/2007
• Issued: 09/06/2011
• Est. Priority Date: 03/23/2007
• Status: Expired due to Fees

• Current Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Original Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.)
• Inventors: Saund, Eric, Ruiz, Jaime G.
• Primary Examiner(s): Le, Brian Q
• Assistant Examiner(s): Park, Edward
• Application Number:
  US11/726,952
• Publication Number:
  US 20080232690A1
• Time in Patent Office: 1,628 Days
• Field of Search: 382/181, 382/187, 382/189
• US Class Current: 382/187
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps

Abstract

A method and system provides intelligent node-link diagram creation and editing, including an electronic display device having a surface on which a writing and/or drawing canvas is provided. An input device permits writing and/or drawing of electronic ink strokes, and a computing device is in operative association with the electronic display and the input device, and has stored therein for operation in connection with actions of the input device, a user interface (UI). The UI includes a graphical user interface (GUI) module, which controls input and display of the ink strokes applied to the canvas by the input device, and a recognition module which interprets the ink strokes by performance of structure recognition operations.

Claims

1. A system for intelligent node-link diagram creation and editing comprising:

an electronic display device having a surface on which a writing and/or drawing canvas is provided;

an input device for writing and/or drawing electronic ink strokes; and

a computing device, in operative association with the electronic display and the input device, and having stored therein for operation in connection with actions by the input device, a user interface (UI) including,

a graphical user interface (GUI) module which controls input and display of the ink strokes applied to the canvas by the input device, and

a recognition module which interprets the ink strokes by performance of structure recognition operations, wherein the structure recognition operations are configured to;

(i) group related ink strokes to form multiple hypotheses reflecting structured interpretations of the ink strokes, wherein the ink strokes are determined to be related based on similarity characteristics, including but not limited to shape similarity and location proximity;

(ii) apply local confidence scores to the multiple hypotheses based on local evidence;

(iii) form a lattice of the hypotheses according to their support relations;

(iv) apply constraints among the hypotheses;

(v) search for hypotheses, under the applied constrains, having collectively high scoring assignments; and

(vi) form a subset hypothesis lattices which represents an interpretation of the node-link diagram using hypotheses with collectively high scoring assignments.

2. The system according to claim 1, wherein the GUI module and recognition module are configured to employ an inferred mode protocol, to address ambiguous ink strokes.

3. The system according to claim 1, further including,

an unprocessed strokes queue for holding unprocessed ink strokes and for passing the unprocessed strokes between the GUI module and the recognition module, and

a processed strokes queue for holding processed ink strokes and for passing the processed strokes between the GUI module and the recognition module.

4. The system according to claim 1, further including,

an interface between the GUI module and recognition module for passing of selection operations and command operations therebetween.

5. The system according to claim 1, wherein the input device configured to enter ink strokes, is further configured to select ink strokes, and specify commands to be performed on selected ink strokes, without selecting a mode of operation prior to entering the ink strokes.

6. The system according to claim 5, further including a pop-up button positioned near an ink stroke input by the input device, when the system determines the meaning of the input ink stroke is ambiguous and selection or non-selection of the pop-up button eliminates the ambiguity.

7. The system according to claim 1, wherein the GUI module and recognition module operate to automatically determine the ink strokes on the canvas as part of a node-link diagram.

8. The system according to claim 1, wherein the node-link diagram is a concept map.

9. A method for intelligent node-link diagrams creation and editing by an electronic processor comprising:

generating a writing and/or drawing canvas on a surface of an electronic display device by the electronic processor;

inputting writing and/or drawing electronic ink strokes by use of an input device;

detecting the input ink strokes by a user interface (UI) stored on the electronic processor;

controlling the input and display of the ink strokes by a graphical user interface (GUI) of the user interface (UI); and

interpreting the ink strokes by use of structured recognition operations, undertaken by a recognition module of the electronic processor, wherein the structure recognition operations are configured to;

(i) group related ink strokes to form multiple hypotheses reflecting structured interpretations of the ink strokes, wherein the ink strokes are determined to be related based on similarity characteristics, including but not limited to shape similarity and location proximity;

(ii) apply local confidence scores to the multiple hypotheses based on local evidence;

(iii) form a lattice of the hypotheses according to their support relations;

(iv) apply constraints among the hypotheses;

(v) search for hypotheses, under the applied constraints, having collectively high scoring assignments; and

(vi) form a subset hypothesis lattice which represents an interpretation of the node-link diagram using hypotheses with collectively high scoring assignments.

10. The method according to claim 9, further including,

applying an inferred mode protocol to address ambiguous ink strokes entered on the canvas by the input device.

11. The method according to claim 9, further including:

providing unprocessed ink strokes to an unprocessed strokes queue;

passing the unprocessed ink strokes on the unprocessed strokes queue between the GUI module and the recognition module;

providing processed ink strokes to a processed strokes queue; and

passing the processed ink strokes between the GUI module and the recognition module.

12. The method according to claim 9, further including,

passing selection operations and command operations between the GUI module and the recognition module by a system interface.

13. The method according to claim 9, wherein in addition to entering the ink strokes by use of the input device, further including:

selecting ink strokes by the input device, and

specifying commands to be performed on selected ink strokes by the input device, wherein the entering, selecting and specifying commands occur without selecting a mode of operation prior to the entering of the ink strokes.

14. The method according to claim 13, further including,

determining an intended meaning of the ink stroke input by the input device is ambiguous;

and

displaying a pop-up button positioned near the ink stroke determined to be ambiguous, wherein selection or non-selection of the pop-up button eliminates the ambiguity.

15. The method according to claim 9, further including,

automatically determining by the system, the ink strokes on the canvas are part of a node-link diagram.

16. The method according to claim 9, wherein the node-link diagram is a concept map.

17. The method according to claim 9, wherein the GUI and Recognition modules operate asynchronously with respect to each other.

18. The method according to claim 9, wherein the intelligent editing includes intelligent selection by use of tapping actions, encircling actions and scratchout actions.