Script character processing method and system with bit-mapped document editing

• US 5,590,257 A
• Filed: 06/14/1993
• Issued: 12/31/1996
• Est. Priority Date: 03/20/1991
• Status: Expired due to Term

• Current Assignee: Microsoft Corporation
• Original Assignee: Mitchell D. Forcier
• Inventors: Forcier, Mitchell D.
• Primary Examiner(s): Herndon, Heather R.
• Assistant Examiner(s): Burwell, Joseph R.
• Application Number:
  US08/077,293
• Time in Patent Office: 1,296 Days
• Field of Search: 395/145, 395/146, 395/148, 382/119, 382/123, 382/179, 382/186, 382/187, 382/188, 382/189
• US Class Current: 715/273
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting

Abstract

A pen-based processor needs to be usable to input and edit script like a text-based computer but retain a resemblance to the user of a pad and pencil. The disclosed system and method implement input, editing and other manipulation of glyphs including handwritten script, ASCII test, bit-mapped images and drawings in a common document, using a compatible internal representation of the data and a simple, consistent set of user control functions. These functions are invoked using an intuitive and interactive set of user gestures which do not distract the user from the task of inputting or editing the document. A two-step gesture method avoids confusion between strokes and command gestures and allows use of similar gestures for different functions within the same and different contexts. The system infers from customary user writing conventions that certain relationships of data are to be preserved and maintains the relationships, subject to user override, during editing. The display document is formatted to contain lined or unlined areas of glyphs that can be edited, such as by mapping each glyph in the image lines to a machine editable line space including insertion of a moving space into pre-existing document text and word wrapping. Adjoining drawing areas are unaffected by editing of text data.

Claims

1. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, and

the step of establishing a baseline comprising;

detecting all non-zero bits in a row which represent visible dots on the image;

counting the detected bits; and

establishing a baseline on the row where the count is less than or equal to a predetermined number of bits.

2. A method according to claim 1 further including repeating the detecting, the counting, and the establishing steps for subsequent rows.

3. A method according to claim 1 wherein the predetermined number of bits is zero.

4. A method according to claim 1 wherein the predetermined number of bits is a user-definable value.

5. A method according to claim 4 wherein the step of determining the height of the image line further includes:

comparing the height of the current image line to a height of the preceding image line;

decomposing the current image line into multiple image lines where the height of the current image line is greater than a predetermined multiple of the height of the preceding image line.

6. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the height of the image line comprises;

determining a baseline for a preceding image line adjacent to the current image line; and

subtracting the baseline of the preceding image line from the baseline of the current image line, the difference being the height of the current image line.

7. A method according to claim 6 wherein the decomposing step comprises:

decomposing the current image line into a number of blank image lines according to the following equation;

number of blank image lines=(height of the current image line/height of preceding image line)-1, wherein each blank image line has a height equal to the height of the preceding image line; and

decomposing the current image line into a visible image line having a height according to the following equation;

height of current visible image line=height of current image line-(number of blank image lines×

height of preceding image line).

8. A method according to claim 6 wherein mapping each glyph in the image lines to a machine editable line space comprises:

mapping each image line to a corresponding machine editable line space;

detecting a glyph in an image line; and

mapping the detected glyph to a position in the corresponding machine editable line space.

9. A method according to claim 8 wherein the detecting step comprises:

detecting an initial visible point; and

detecting all points contiguous with the initial visible point.

10. A method according to claim 9 further including extracting a rectangular bitmap circumscribing the detected points.

11. A method according to claim 9 further including repeating the detecting steps for a subsequent glyph.

12. A method according to claim 8 wherein the mapping step comprises:

assigning a horizontal displacement to the glyph in the machine editable line space corresponding to a horizontal position of the glyph in the corresponding image line; and

assigning a vertical displacement to the glyph in the machine editable line space corresponding to a vertical position of the glyph in the corresponding image line.

13. A method according to claim 12 wherein the mapping step further includes assigning an origin to the glyph including:

an x-coordinate equal to a displacement in bits relative to a left-margin of the image; and

a y-coordinate equal to a displacement in bits relative to machine editable line space.

14. A method according to claim 12 wherein the origin corresponds to a lower left corner of the glyph.

15. A method according to claim 6 further including inserting a beginning-of-line (BOL) marker in each line space.

16. A method according to claim 6 further comprising:

entering a first command using the input device at a user-determined location along a selected one of said line spaces to assert an insert moving space function, the computer and document display being responsive to open a predetermined amount of added space at said location in the selected line space including shifting any pre-existing glyphs in the selected line space on one side of said location in a direction along the line away from said location;

establishing a current input line in the selected line space;

entering additional strokes on the display other than in the moving space;

moving the input device into the moving space; and

establishing the moving space line as the current input line.

17. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the beginning point comprises;

detecting a left-most non-zero bit in the current image line; and

setting the left margin equal to the detected left-most bit.

18. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the beginning point comprises;

detecting a left-most non-zero bit in the current image line;

detecting a left-most non-zero bit in a subsequent image line; and

setting the left margin equal to the detected left-most bit of the subsequent image line where the left-most bit of the subsequent image line is to the left of the left-most bit of the current image line.

19. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of file current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the beginning point comprises;

detecting a left-most non-zero bit in the current image line;

detecting a left-most non-zero bit in a preceding image line; and

setting the left margin equal to the detected left-most bit of the preceding image line where the left-most bit of the current image line is within a predetermined number of bits from the detected bit of the current image line.

20. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the beginning point comprises;

detecting a left-most non-zero bit in the current image line;

detecting a left-most non-zero bit in a preceding image line; and

setting the left margin equal to the detected left-most bit of the current image line where the left-most bit of the current image line is to the left of the left-most bit of the preceding image line.

21. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

the step of decomposing the image including;

establishing a baseline for a current image line;

determining a height of the current image line; and

determining beginning and end points of the current image line relative to the image;

the step of mapping including defining the machine editable line space according to the baseline, height, and beginning and end points, wherein the step of determining the beginning point comprises;

detecting a left-most non-zero bit in the current image line;

detecting a left-most non-zero bit in a following image line; and

setting the left margin equal to the detected left-most bit of the current image line where the left-most bit of the current image line is to the left of the left-most bit of the following image line.

22. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

wherein mapping each glyph in the image lines to a machine editable line space comprises;

mapping each image line to a corresponding machine editable line space;

detecting a glyph in an image line; and

mapping the detected glyph to a position in the corresponding machine editable line space;

a height of each machine editable line space is equal to a height of the corresponding image line.

23. A pen-based computer method for a user interactively to edit a series of glyphs which include one or more of handwritten script, text or bitmapped images using a pen-type input device on a document display having a series of lines defining machine editable line spaces for writing script, the method comprising:

inputting and displaying a bit-mapped image containing a series of glyphs arranged in a generally linear format but initially unconstrained to the line spaces;

decomposing the image into image lines; and

mapping each glyph in the image lines to one of the machine editable line spaces;

wherein mapping each glyph in the image lines to a machine editable line space comprises;

mapping each image line to a corresponding machine editable line space;

detecting a glyph in an image line; and

mapping the detected glyph to a position in the corresponding machine editable line space;

further including repositioning the machine-editable line spaces relative to the display.

24. A method according to claim 23 wherein the repositioning step includes vertically offsetting the machine editable line spaces relative to the display.

25. A method according to claim 23 wherein the repositioning step includes assigning a left and a right margin to the machine editable line spaces.

26. A pen-based computer method for a user interactively to edit glyphs on a lineless document display to insert handwritten script using a pen-type input device, the method comprising:

forming an input area on the display for receiving the handwritten script;

entering a series of strokes in a predetermined direction within the input area;

entering a first command using the input device at a user-determined location along the series to assert an insert moving space function, the computer and document display being responsive to open a predetermined amount of added space at said location including shifting any pre-existing glyphs in the series on one side of said location in a direction along the series away from said location; and

entering handwritten script in the added space using the input device, the computer and document display being responsive to the entry of the script to supplement the added space to approximate the amount of added script;

detecting a center of gravity of a current stroke in the series;

detecting a center of gravity of a subsequent stroke in the series;

comparing the center of gravity of the subsequent stroke to the center of gravity of the current stroke;

associating the current stroke with a current line; and

associating the subsequent stroke with a subsequent line if the center of gravity of the subsequent stroke differs from the center of gravity of the current stroke by a user-definable distance.

27. A method according to claim 26 further including associating all of the strokes entered prior to the subsequent stroke with the current line.

28. A method according to claim 27 further including:

detecting visible bits for each stroke associated with the current line; and

determining a current line vector based on the detected visible bits, the current line vector being a best-fit linear approximation of the detected visible bits.

29. A method according to claim 28 further including:

decomposing the strokes associated with the current line into words using breakpoint analysis; and

associating the words with the current line vector.

30. A method according to claim 29 wherein the breakpoint analysis step includes forming a parallelogram having top and bottom edges parallel to the current vector line.

31. A method according to claim 30 further including repositioning the words relative to the current line vector, the repositioned words being generally collinear with the current line vector.

32. A method according to claim 28 further including realigning the current line vector parallel to the display.

33. A method according to claim 26 further comprising:

determining a mid-point boundary line between the current line vector and the subsequent vector line;

associating line space between the boundary line and the current line vector with the current line vector; and

associating line space between the boundary line and the subsequent line vector with the subsequent line vector.

34. A method according to claim 26 wherein entering a series of strokes includes:

entering a first series of strokes in a first column;

entering a second series of strokes in a second column; and

wherein the amount of added space exceeds an available space in the first column, the computer and document display being responsive to wrap a word in the first column to the second column.

35. A method according to claim 34 in which the amount of added space does not exceed an available space in the first column, the computer and document display being responsive to wrap a word from a present line in the first column to a subsequent line in the first column.

36. A method according to claim 28 including:

determining a length of the stroke; and

rooting the stroke in the line space in which the center of gravity resides.

37. A method according to claim 36 further comprising:

determining a length of the stroke to define the stroke as a long stroke or a short stroke;

rooting the stroke in a line space in which the input device first touched down on the display if the stroke is a short stroke; and

rooting the stroke in the line space in which the center of gravity resides if the stroke is a long stroke.

38. A method according to claim 36 further comprising:

examining N consecutive strokes;

determining a common input line space; and

establishing the common input line space as the current input line space if;

a) the N strokes are within a user-definable horizontal distance,

b) the N strokes touch the common input line space, and

c) the N strokes are contained within three adjacent line spaces including the common input line space, a preceding line space, and a subsequent line space.

39. A method according to claim 38 wherein the step of determining the common input line space comprises:

determining a vertical center of gravity for each of the N strokes;

determining for each stroke a resident line wherein the center of gravity of said stroke resides; and

establishing the common input line as a resident line having a user-definable percentage of the N center of gravities.

40. A method according to claim 38 wherein the step of determining the common input line space comprises:

determining a vertical center of gravity for each of the N strokes;

determining an average center of gravity from the N vertical center of gravities; and

establishing the common input line as a line wherein the average center of gravity resides.