System and method for superimposed handwriting recognition technology

• US 9,384,403 B2
• Filed: 07/10/2015
• Issued: 07/05/2016
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Myscript
• Original Assignee: Myscript
• Inventors: Wimmer, Zsolt, Perraud, Freddy, Lallican, Pierre-Michel, Aradilla, Guillermo
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/796,092
• Publication Number:
  US 20150356360A1
• Time in Patent Office: 361 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/2268 using stroke segmentation
• CPC Class Codes: G06V 30/2272 with lexical matching
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/32 Digital ink

Abstract

A system and method that is able to recognize a user'"'"'s natural superimposed handwriting without any explicit separation between characters. The system and method is able to process single-stroke and multi-stroke characters. It can also process cursive handwriting. Further, the method and system can determine the boundaries of input words either by the use of a specific user input gesture or by detecting the word boundaries based on language characteristics and properties. The system and method analyzes the handwriting input through the processes of fragmentation, segmentation, character recognition, and language modeling. At least some of these processes occur concurrently through the use of dynamic programming.

Claims

1. A non-transitory computer readable medium having computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for providing handwriting recognition for a plurality of fragments of input strokes at least partially superimposed on one another, said method comprising:

detecting relative positions of input strokes of at least two sequential fragments of the input strokes;

detecting the geometry of the input strokes of the at least two sequential fragments;

determining from the detected relative positions the superimposition of segments of the input strokes and determining from the detected geometry whether the superimposed segments likely form a character;

classifying the fragments based on the determined likely characters; and

providing the classified fragments to a recognition engine for evaluation of character hypotheses based on the classified fragments,

wherein, within the recognition engine, the method comprises;

creating a segmentation graph based on the strokes of the classified fragments, wherein the segmentation graph consists of nodes corresponding to character hypotheses;

assigning a recognition score to each node of the segmentation graph based on a pattern classifier;

generating linguistic meaning of the input strokes based on the recognition scores and a language model; and

providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

2. A non-transitory computer readable medium according to claim 1, wherein the input strokes are preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input strokes.

3. A non-transitory computer readable medium according to claim 1, wherein each classified fragment is defined to contain complete characters formed by the input strokes of one or more input fragments.

7. A non-transitory computer readable medium according to claim 1, wherein the detected relative positions of the input strokes of the at least sequential fragments are detected from both spatial and temporal information of the input strokes.

8. A non-transitory computer readable medium according to claim 1, wherein the segmentation graph further includes nodes corresponding to space hypotheses between the character hypotheses based on the classified fragments.

4. A method for providing handwriting recognition for a superimposed input stroke, said method comprising:

detecting relative positions of input strokes of at least two sequential fragments of the input strokes;

detecting the geometry of the input strokes of the at least two sequential fragments;

determining from the detected relative positions the superimposition of segments of the input strokes and determining from the detected geometry whether the superimposed segments likely form a character;

classifying the fragments based on the determined likely characters; and

providing the classified fragments to a recognition engine for evaluation of character hypotheses based on the classified fragments,

wherein, within the recognition engine, the method comprises;

creating a segmentation graph based on the strokes of the classified fragments, wherein the segmentation graph consists of nodes corresponding to character hypotheses;

assigning a recognition score to each node of the segmentation graph based on a pattern classifier;

generating linguistic meaning of the input strokes based on the recognition scores and a language model; and

providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

5. A method according to claim 4, wherein the input strokes are preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input strokes.

6. A method according to claim 4, wherein each classified fragment is defined to contain complete characters formed by the input strokes of one or more input fragments.

9. A method according to claim 4, wherein the detected relative positions of the input strokes of the at least sequential fragments are detected from both spatial and temporal information of the input strokes.

10. A method according to claim 4, wherein the segmentation graph further includes nodes corresponding to space hypotheses between the character hypotheses based on the classified fragments.