System and method for editing ink objects

• US 7,503,015 B2
• Filed: 03/30/2005
• Issued: 03/10/2009
• Est. Priority Date: 09/21/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Liu, Youjun, Li, Yantao, Wang, Xianfang, Chen, Tongxian, Wang, Jian
• Primary Examiner(s): Bashore; William L
• Assistant Examiner(s): ZAHR, ASHRAF A
• Application Number:
  US11/095,994
• Publication Number:
  US 20060061779A1
• Time in Patent Office: 1,441 Days
• Field of Search: 715 50-542, 715/863, 345173-178, 382213-215
• US Class Current: 715/863
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A system and method for editing ink objects recognized in ink input is provided. An ink parser may recognize an ink object in ink input and then an ink editing user interface may edit the ink object recognized by the ink parser. The ink parser may include a chart detector, shape recognizer, and various ink object recognizers such as a chart recognizer, a list detector and a table detector. The various ink object recognizers may recognize particular ink objects. The ink editing user interface may edit the ink object recognized by the ink parser. The ink editing user interface may include a chart editor, list editor, table editor, mode switcher, and a visualizer. The mode switcher may switch the ink editing system between inking mode and ink editing mode.

Claims

1. A computer system for editing ink input, the ink input comprising one or more ink strokes, the system comprising:

a processor;

a memory;

an ink parser for recognizing an ink object in ink input within a software application, the ink object recognition comprising;

generating, as part of the software application, an image grid comprising a plurality of grid spaces, the image grid encompassing each of the one or more ink strokes in the ink object, the image grid providing a reference for determining the stroke positions of one or more recognized ink objects;

positioning the strokes of the ink object on the generated image grid;

marking each grid space that is intersected by ink strokes as an intersected grid space, such that the general shape of the ink strokes is identifiable using the intersected grid spaces; and

flood-filling each non-marked space in the image grid with whitespace;

an ink editing user interface operably coupled to the ink parser for editing the ink object recognized by the ink parser, wherein the ink editing operation comprises the following;

repositioning at least one recognized container according to a user input indicating that at least one of the containers is to be repositioned;

determining that the container that is to be repositioned is connected to at least one other container with one or more connectors;

determining a connector type for each of the one or more connectors; and

based on the determined connector type, reflowing the one or more connectors linking the recognized, repositioned containers in an appropriate manner for the determined connector type, such that the connector appropriately links the repositioned container and the other container; and

a mode recognizer operably coupled to the ink editing user interface for switching between inking mode and ink editing mode.

2. The system of claim 1 wherein the ink editing user interface comprises an operably coupled chart editor for editing a hand-drawn chart recognized by the ink parser.

3. The system of claim 1 wherein the ink editing user interface comprises an operably coupled list editor for editing a hand-drawn list recognized by the ink parser.

4. The system of claim 1 wherein the ink editing user interface comprises an operably coupled table editor for editing a hand-drawn table recognized by the ink parser.

5. The system of claim 1 wherein the ink editing user interface comprises an operably coupled visualizer for displaying a chart representing the information in a hand-drawn table recognized by the ink parser.

6. The system of claim 1 wherein the ink parser comprises an operably coupled chart detector having a container detector for detecting the containers within ink input and a connector detector for detecting the connectors within ink input.

7. The system of claim 1 wherein the ink parser comprises an operably coupled shape recognizer having a container recognizer and a connector recognizer.

8. The system of claim 1 wherein the ink parser comprises:

a chart recognizer for recognizing a hand-drawn chart in ink input;

a list detector for recognizing a hand-drawn list in ink input; and

a table detector for recognizing a hand-drawn table in ink input.

9. A method in a computer system for editing ink input, the method comprising:

receiving ink input comprising one or more ink strokes;

parsing the ink input;

performing ink object recognition within a software application to recognize an ink object among the one or more strokes of the ink input, the ink object recognition comprising;

generating, as a part of the software application, an image grid comprising a plurality of grid spaces, the image grid encompassing each of the one or more ink strokes in the ink object, the image grid providing a reference for determining the stroke positions of one or more recognized ink objects;

positioning the strokes of the ink object on the generated image grid;

marking each grid space that is intersected by ink strokes as an intersected grid space, such that the general shape of the ink strokes is identifiable using the intersected grid spaces; and

flood-filling each non-marked space in the image grid with whitespace;

switching to an editing mode to edit the ink object;

editing the ink object by performing an ink editing operation, wherein the ink editing operation comprises the following;

repositioning at least one recognized container according to a user input indicating that at least one of the containers is to be repositioned;

determining that the container that is to be repositioned is connected to at least one other container with one or more connectors;

determining a connector type for each of the one or more connectors; and

based on the determined connector type, reflowing the one or more connectors linking the recognized, repositioned containers in an appropriate manner for the determined connector type, such that the connector appropriately links the repositioned container and the other container; and

regenerating the ink representing the edited ink object.

10. The method of claim 9 further comprising generating a drawing of the ink object recognized in the ink input.

11. The method of claim 9 wherein performing ink object recognition for an ink object in the ink input comprises recognizing a chart and wherein editing the ink object by performing the ink editing operation comprises editing the recognized chart.

12. The method of claim 9 wherein performing ink object recognition for an ink object in the ink input comprises recognizing a list and wherein editing the ink object by performing the ink editing operation comprises editing the recognized list.

13. The method of claim 9 wherein performing ink object recognition for an ink object in the ink input comprises recognizing a table and wherein editing the ink object by performing the ink editing operation comprises editing the recognized table.

14. The method of claim 9 wherein editing the ink object comprises creating a bounding box around the ink object and displaying a selection handle associated with the bounding box to resize the ink object.

15. The method of claim 9 wherein editing the ink object comprises creating a bounding box around the ink object and displaying a selection handle associated with the bounding box to rotate the ink object.

16. A computer-readable storage medium having computer-executable instructions for performing the method of claim 9.

17. The method of claim 9, further comprising:

determining that the connector is a normal connector; and

reflowing the connector such that the connector'"'"'s line segments comprise horizontal and vertical lines.

18. The method of claim 9, further comprising:

determining that the connector is a common connector; and

reflowing the connector such that the connector'"'"'s line segments are deformed to allow for a lengthened or shortened curve while preserving the connector'"'"'s curve.

19. The method of claim 9, further comprising:

determining that the connector is a straight connector with a directional arrow pointing to the repositioned container; and

reflowing the connector such that the connector'"'"'s directional arrow continues to point to the repositioned container, regardless of where the repositioned container is repositioned.

20. A computer system for editing ink input, comprising:

means for receiving ink input, the ink input comprising one or more ink strokes;

means for parsing the ink input;

means for performing ink object recognition within the a software application to recognize an ink object among the one or more strokes of the ink input, the ink object recognition comprising;

generating, as a part of the software application, an image grid comprising a plurality of grid spaces, the image grid encompassing each of the one or more ink strokes in the ink object, the image grid providing a reference for determining the stroke positions of one or more recognized ink objects;

positioning the strokes of the ink object on the generated image grid;

marking each grid space that is intersected by ink strokes as an intersected grid space, such that the general shape of the ink strokes is identifiable using the intersected grid spaces; and

flood-filling each non-marked space in the image grid with whitespace;

means for switching to an editing mode to edit the ink object;

means for editing the ink object by performing an ink editing operation, wherein the ink editing operation comprises the following;

repositioning at least one recognized container according to a user input indicating that at least one of the containers is to be repositioned;

determining that the container that is to be repositioned is connected to at least one other container with one or more connectors;

determining a connector type for each of the one or more connectors; and

based on the determined connector type, reflowing the one or more connectors linking the recognized, repositioned containers in an appropriate manner for the determined connector type, such that the connector appropriately links the repositioned container and the other container; and

means for regenerating the ink representing the edited ink object.