System, method and apparatus for automatic segmentation and analysis of ink stream

• US 7,925,088 B2
• Filed: 02/23/2005
• Issued: 04/12/2011
• Est. Priority Date: 02/23/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Bhaskarabhatla, Ajay
• Primary Examiner(s): Couso; Jose L
• Application Number:
  US11/884,747
• Publication Number:
  US 20080159625A1
• Time in Patent Office: 2,239 Days
• Field of Search: 382/173, 382/177, 382/179, 382/181, 382186-188, 382/200, 382202-203, 382/256, 382312-316, 178 1801- 1811, 345156-158, 345162-167, 345179-183, 345/441, 345/467, 345/469, 715/863
• US Class Current: 382/179
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction

Abstract

A technique that provides for real-time segmentation of hand written traces during data entry into a computer. In one example embodiment, this is achieved by computing a current trace bounding contour as a function of a drawn current trace. A current selection bounding contour is then computed as a function of the drawn current trace and the current trace bounding contour. The current selection bounding contour includes one or more previously drawn traces. The computed current trace bounding contour is then compared with the computed current selection bounding contour. The hand written traces including the current trace and the one or more previously drawn traces are then dynamically segmented as a function of the comparison.

Claims

1. A method comprising:

drawing a current trace using a user input device;

computing a current trace bounding contour as a function of the drawn current trace;

dynamically displaying the drawn current trace on an output device;

computing a current selection bounding contour as a function of the drawn current trace and the current trace bounding contour, wherein the current selection bounding contour includes one or more previously drawn traces;

comparing the computed current trace bounding contour with the computed current selection bounding contour; and

real-time segmenting of an ink stream including the current trace and the one or more previously drawn traces as a function of the comparison.

2. The method of claim 1, further comprising:

repeating the computing, comparing, and real-time steps for a next drawn trace.

3. The method of claim 1, wherein computing the current trace bounding contour as a function of the drawn current trace comprises:

initializing the current trace bounding contour upon occurrence of a pen-down event by the user input device comprising a stylus and a digitizer;

drawing the current trace using the stylus on the digitizer; and

dynamically computing and updating the initialized current trace bounding contour along with the drawing of the current trace as a function of the drawn current trace.

4. The method of claim 3, wherein computing the current selection bounding contour as a function of the drawn current trace and the current trace bounding contour comprises:

determining whether the current trace is a first trace in the ink stream being drawn upon occurring of a pen-up event by the stylus; and

if so, then selecting the drawn current trace, initializing the current selection bounding contour and computing the current selection bounding contour as a function of the current trace and the current trace bounding contour.

5. The method of claim 4, further comprising:

displaying the drawn current trace along with the current selection bounding contour on the output device comprising the display of the digitizer.

15. The method of claim 14, further comprising:

if so, then displaying the one or more previously drawn traces along with a previously formed selection bounding contour substantially surrounding the one or more previously drawn traces;

removing the previously drawn selection bounding contour from the one or more previously drawn traces;

displaying the one or more previously drawn traces in a third contrasting color to indicate that the one or more previously drawn traces are part of the current segment;

storing the one or more previously drawn traces along with the current selection bounding contour; and

and repeating the steps of claim 1 for the drawn current trace to start formation of a next segment.

6. A method for real-time segmentation of a drawn ink stream comprising:

initializing a current trace bounding contour upon occurrence of a pen-down event by a stylus on a digitizer;

drawing a current trace using the stylus on the digitizer;

dynamically computing and updating the initialized current trace bounding contour along with the drawing of the current trace as a function of the drawn current trace;

dynamically displaying the drawn current trace along with the computed and updated current trace bounding contour substantially simultaneously on the digitizer;

determining whether the current trace is a first trace in the drawn ink stream upon occurring of a pen-up event by the stylus;

if so, then selecting the drawn current trace, initializing a current selection bounding contour, computing the current selection bounding contour as a function of the current trace and the current trace bounding contour and replacing the current trace bounding contour with the current selection bounding contour; and

displaying the drawn current trace along with the current selection bounding contour on the digitizer.

7. The method of claim 6, further comprising:

if not, then computing spatial parameters between the current trace bounding contour and the current selection bounding contour;

computing and updating the current selection bounding contour as a function of the computed spatial parameters;

removing the current trace bounding contour from the digitizer; and

displaying the drawn current trace and the one or more previously drawn traces along with the updated current selection bounding contour.

8. The method of claim 7, wherein the spatial parameters comprise parameters selected from the group comprising of overlap and nearness.

9. The method of claim 7, wherein dynamically updating the initialized current trace bounding contour along with the drawing of the current trace comprises:

dynamically updating the current trace bounding contour along with the drawing of the current trace such that the current trace bounding contour substantially surrounds the current trace.

10. The method of claim 7, wherein, in displaying, the updated current selection bounding contour substantially surrounds the current trace along with the one or more previously drawn traces.

11. The method of claim 7, further comprising:

determining whether there is another trace in the ink stream that needs to be drawn on the digitizer;

if so, then repeating the above computing spatial parameters, removing, and displaying steps; and

if not, then storing the drawn current trace and the one or more previously drawn traces along with the current selection bounding contour.

12. The method of claim 6, wherein dynamically displaying the updated current trace bounding contour comprises:

dynamically displaying the updated current trace bounding contour in a first contrasting color on the digitizer.

13. The method of claim 6, wherein displaying the selection bounding contour on the digitizer comprises:

displaying the selection bounding contour in a second contrasting color.

14. The method of claim 7, wherein displaying the drawn current trace and the one or more previously drawn traces along with the updated current selection bounding contour comprises:

determining whether the computed spatial parameters for the drawn current trace is greater a threshold value; and

if not, then displaying the drawn current trace and the one or more previously drawn traces along with the current selection bounding contour.

16. The method of claim 7, wherein the one or more traces comprise traces associated with ink stream categories selected from the group consisting of drawing, musical tone, text, mathematical formula, graph, and symbol.

17. The method of claim 7, wherein, in computing, the spatial parameters are configurable by a user to facilitate entry of traces associated with ink stream categories selected from the group consisting of drawing, musical tone, text, symbol, mathematical formula, and graph.

18. The method of claim 6, wherein displaying the drawn current trace along with the selection bounding contour on the digitizer indicates a selection of the drawn trace.

19. The method of claim 6, wherein, in displaying, the selection bounding contour substantially surrounds the drawn current trace to provide a visual feedback of the boundary of the selected drawn current trace and to assist the user in drawing a subsequent trace so that the subsequent trace does not overlap the selected drawn current trace.

20. An article comprising:

a non-transitory computer readable storage medium having instructions that, when executed by a computing platform, result in execution of a method comprising;

computing a current trace bounding contour as a function of a drawn current trace;

computing a current selection bounding contour as a function of the drawn current trace and the current trace bounding contour, wherein the current selection bounding contour includes one or more previously drawn traces;

comparing the computed current trace bounding contour with the computed current selection bounding contour; and

real-time segmenting a drawn ink stream including the current trace and the one or more previously drawn traces as a function of the comparison.

21. The article of claim 20, further comprising:

repeating the computing, comparing, and real-time steps for a next drawn trace.

22. The article of claim 20, wherein computing the current trace bounding contour as a function of the drawn current trace comprises:

initializing the current trace bounding contour upon occurring of a pen-down event by a stylus on a digitizer;

drawing the current trace using the stylus on the digitizer; and

dynamically computing and updating the initialized current trace bounding contour along with the drawing of the current trace as a function of the drawn current trace.

23. The article of claim 22, further comprising:

dynamically displaying the drawn current trace.

24. The article of claim 22, wherein computing the current selection bounding contour as a function of the drawn current trace and the current trace bounding contour comprises:

determining whether the current trace is a first trace in the drawn ink stream upon occurrence of a pen-up event by the stylus; and

if so, then selecting the drawn current trace, initializing the current selection bounding contour and computing the current selection bounding contour as a function of the current trace and the current trace bounding contour.

25. The article of claim 24, further comprising:

displaying the drawn current trace along with the current selection bounding contour on the digitizer.

26. An apparatus for segmentation of inputted ink stream comprising:

a digitizer and a stylus; and

a segmentation engine implemented on a computing device coupled to the digitizer that is responsive to a pen-down event, the stylus on the digitizer, and a pen-up event, wherein the segmentation engine initializes a current trace bounding contour upon the pen-down event by the stylus on the digitizer,

wherein a current trace is drawn using the stylus on the digitizer, wherein the initialized current trace bounding contour is dynamically computed and updated as a function of the drawn current trace by the segmentation engine, wherein the drawn current trace is dynamically displayed on the digitizer,

wherein the segmentation engine determines whether the current trace is a first trace in the drawn ink stream upon the pen-up event by the stylus, wherein the segmentation engine selects the drawn current trace, initializes a current selection bounding contour, and computes the current selection bounding contour as a function of the current trace and the trace bounding contour and replaces the trace bounding contour with the current selection bounding contour if the drawn current trace is the first trace, and

wherein the segmentation engine displays the current trace along with the current selection bounding contour on the digitizer.

27. The apparatus of claim 26, wherein the segmentation engine computes one or more spatial parameters between the current trace bounding contour and the current selection bounding contour,

wherein the segmentation engine computes and updates the current selection bounding contour as a function of the one or more computed spatial parameters, and

wherein the segmentation engine displays the drawn current trace and the one or more previously drawn traces along with the updated current selection bounding contour indicating that the selection of the current trace and the one or more previously drawn traces are part of a current segment.

28. The apparatus of claim 27, wherein the segmentation engine determines whether the one or more computed spatial parameters is greater than a threshold value,

wherein the segmentation engine displays the current trace and the one or more previously drawn traces along with the current selection bounding contour indicating that the current trace and the one or more previously drawn traces are part of the current segment if the one or more computed spatial parameters are not greater than the threshold value, and

wherein the segmentation engine replaces the current selection bounding contour with a next selection bounding contour indicating a start of a next segment and removes the current selection bounding contour from the one or more previously drawn traces and displays the one or more previously drawn traces in a first contrasting color and stores the one or more previously drawn traces as a current segment if the one or more computed spatial parameters are greater than the threshold value.

29. A computer system for triggering handwriting recognition comprising:

a network interface;

an input module coupled to the network interface that receives the input data via the network interface;

a processing unit;

a memory coupled to the processing unit;

a user input device;

an output device; and

a segmentation engine stored in the memory coupled to the user input device and that is responsive to a pen-down event, a stylus on a digitizer of the user input device, and a pen-up event, wherein the segmentation engine initializes a current trace bounding contour upon the pen-down event by the stylus on the digitizer,

wherein a current trace is drawn using the stylus on the digitizer, wherein the initialized current trace bounding contour is dynamically computed and updated as a function of the drawn current trace by the segmentation engine, wherein the drawn current trace is dynamically displayed on the output device,

wherein the segmentation engine determines whether the current trace is a first trace in the drawn ink stream upon the pen-up event by the stylus, wherein the segmentation engine selects the drawn current trace, initializes a current selection bounding contour, and computes the current selection bounding contour as a function of the current trace and the trace bounding contour and replaces the trace bounding contour with the current selection bounding contour if the drawn current trace is the first trace, and

wherein the output device displays the current trace along with the current selection bounding contour on the output device.

30. The system of claim 29, wherein the segmentation engine computes one or more spatial parameters between the current trace bounding contour and the current selection bounding contour,

wherein the segmentation engine computes and updates the current selection bounding contour as a function of the one or more computed spatial parameters, and

wherein the output device displays the drawn current trace and the one or more previously drawn traces along with the updated current selection bounding contour indicating that the selection of the current trace and the one or more previously drawn traces are part of a current segment.

31. The system of claim 30, further comprising:

a handwriting recognition engine coupled to the segmentation engine, wherein the segmentation engine determines whether the one or more computed spatial parameters is greater than a threshold value,

wherein the output device displays the current trace and the one or more previously drawn traces along with the current selection bounding contour indicating that the current trace and the one or more previously drawn traces are part of the current segment if the one or more computed spatial parameters is not greater than the threshold value, and

wherein the segmentation engine replaces the current selection bounding contour with a next selection bounding contour indicating the start of a next segment and removes the current selection bounding contour from the one or more previously drawn traces, and wherein the output device displays the one or more previously drawn traces in a first contrasting color,

wherein the segmentation engine stores the one or more previously drawn traces as a current segment and inputs the one or more previously drawn traces into the handwriting recognition engine for character recognition if the one or more computed spatial parameters are greater than the threshold value.